-- Logs begin at Wed 2022-07-20 07:42:07 UTC, end at Wed 2022-07-20 08:47:53 UTC. -- Jul 20 08:09:38.633608 dv-cinder-823089-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 08:09:38.642063 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 08:09:38.646321 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 08:09:38 2022] *** Jul 20 08:09:38.646321 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 08:09:38.646321 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 08:09:38.646321 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: nodename: dv-cinder-823089-3-iscsi Jul 20 08:09:38.646321 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: machine: x86_64 Jul 20 08:09:38.646321 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: clock source: unix Jul 20 08:09:38.646321 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: pcre jit disabled Jul 20 08:09:38.646321 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: detected number of CPU cores: 4 Jul 20 08:09:38.646321 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: current working directory: / Jul 20 08:09:38.646321 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: detected binary path: /usr/bin/uwsgi-core Jul 20 08:09:38.646321 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: your processes number limit is 47680 Jul 20 08:09:38.647398 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: your memory page size is 4096 bytes Jul 20 08:09:38.647398 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: detected max file descriptor number: 2048 Jul 20 08:09:38.647398 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: lock engine: pthread robust mutexes Jul 20 08:09:38.647398 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: thunder lock: enabled Jul 20 08:09:38.647398 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 08:09:38.647398 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 08:09:38.698611 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: Python main interpreter initialized at 0x55c2269e99c0 Jul 20 08:09:38.698611 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: python threads support enabled Jul 20 08:09:38.698611 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: your server socket listen backlog is limited to 100 connections Jul 20 08:09:38.698611 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: your mercy for graceful operations on workers is 90 seconds Jul 20 08:09:38.699130 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 08:09:38.699130 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: *** Operational MODE: preforking *** Jul 20 08:09:38.699531 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: *** uWSGI is running in multiple interpreter mode *** Jul 20 08:09:38.699531 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: spawned uWSGI master process (pid: 80216) Jul 20 08:09:38.699790 dv-cinder-823089-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 08:09:38.704184 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: spawned uWSGI worker 1 (pid: 80225, cores: 1) Jul 20 08:09:38.704184 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: spawned uWSGI worker 2 (pid: 80226, cores: 1) Jul 20 08:09:38.704184 dv-cinder-823089-3-iscsi devstack@keystone.service[80216]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 08:09:45.134351 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 08:09:45.134351 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: from cryptography.utils import int_from_bytes Jul 20 08:09:45.135621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 08:09:45.135621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: from cryptography.utils import int_from_bytes Jul 20 08:09:45.188500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 08:09:45.188813 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 08:09:45.189099 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] command line args: [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 08:09:45.189475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 08:09:45.189746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ================================================================================ {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 08:09:45.190119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] admin_token = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.190478 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] config_dir = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.190823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.191101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] config_source = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.191392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] debug = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.191704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.192396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.192588 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.192966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.193337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.193703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.194012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_config_append = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.194325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.194653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_dir = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.194959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_file = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.195329 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.195702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.196062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.196936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.197245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.197512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.197860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.198203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.198538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.198917 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.199263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.199604 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.199971 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.200394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.200855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.201511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.201961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] publish_errors = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.202263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.202660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.202973 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.203353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.203718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.204092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] standard_threads = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.204453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.204832 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.205135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.205435 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] use_journal = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.205733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] use_json = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.206039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] use_stderr = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.206335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] use_syslog = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.206750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.207179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.207546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.207934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.208361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.208740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.209156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.209539 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.209920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.210299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.210731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.211114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.211489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.212856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.213355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.213727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.214108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.214466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.214900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.215275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.215658 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.216024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.216421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.216817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.217201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.217574 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.217935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.218295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.218700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.219075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.219456 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.219830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.220189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.220549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.220938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.221330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.221701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.222079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.222471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.222913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.223257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.223584 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.223912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.224238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.224575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.224916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.225263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.225598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.225934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.226266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.226646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.226998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.227396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.227835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.228283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.229145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.229715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.230310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.230651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.231101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.231583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.232022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.232618 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.232948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.233596 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.234043 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.234473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.234967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.235417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.235913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.236530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.237002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.237726 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.238181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.238539 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.239198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.239670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.240122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.240576 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.240964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.241484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.241935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.242387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.242877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.244093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.244550 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.245107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.245578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.246035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.246505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.246981 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.247425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.248069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.248514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.248959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.249489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.250023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.250465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.250957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.251414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.252008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.252471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.252964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.253454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.253896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.254446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.external = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.255097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.255915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.256386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.256959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.password = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.257212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 08:09:45.257212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: from cryptography.utils import int_from_bytes Jul 20 08:09:45.258079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] auth.token = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.258198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 08:09:45.258198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: from cryptography.utils import int_from_bytes Jul 20 08:09:45.258394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.258394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.259162 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.259619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.260194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.260695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.261220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.261781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.262191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.262708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.263071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.263457 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.264019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.264790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.265472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.266057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.266734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.267515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.268232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.268778 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.269472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.270241 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.271021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.271896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.272601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.273311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.273917 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.275290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.276344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.276803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.277257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.277685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.278380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.278767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.279226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.279682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.280859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.281341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.281926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.282386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.284654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.284654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.284654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.284654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.286189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.286189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.286762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.286762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.287755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.288187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.288666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.289175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.289793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.290442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.291040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.291962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.292616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.293078 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.293545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.293992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.294382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.297345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.297345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.297345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.297345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.297345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.297345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.298664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.298905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.299436 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.299826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.300355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.300746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.301215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.301618 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.302034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.302513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.303068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.303535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.304126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.304603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.304987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.305876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.306267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.306687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.307074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.307445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.307825 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.308239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.308646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.309029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.309413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.309798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.310168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.310529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.310950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.311178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 08:09:45.311369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.311530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 08:09:45.311530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] command line args: [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 08:09:45.311812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.311953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 08:09:45.312098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ================================================================================ {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 08:09:45.312239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.312393 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] admin_token = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.312595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] config_dir = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.312737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.312870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.313012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] config_source = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.313150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.313302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] debug = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.313503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.313672 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.313813 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.313954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.313954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.314220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.314363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.314504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.314702 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.314702 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_config_append = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.314964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.315123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.315123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_dir = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.315372 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_file = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.315502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.315643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.315784 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.315938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.316067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.316202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.316354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.316535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.316683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.316828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.316970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.317110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.317252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.317401 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.317557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.317689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.317902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.318052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.318306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.318522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.318770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.318925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.319145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.319294 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.319425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.319601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.319957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.320297 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.320442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.320624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] publish_errors = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.320770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.320904 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.321049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.321714 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.321834 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.322111 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.322392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.322392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.322856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.323066 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] standard_threads = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.323066 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.323463 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.323677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.323677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.323677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] use_journal = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.324263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.324476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] use_json = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.324476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] use_stderr = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.324876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.325079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] use_syslog = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.325079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 08:09:45.325475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.325681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.325879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.326081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.326081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.326475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.326712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.326712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.327111 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.327324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.327324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.327734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.327947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.328147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.328358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.328358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.328732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.328954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.329154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.329363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.329363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.329758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] role.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.329981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.329981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.330390 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] role.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.330623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.330826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.331041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.331041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.331431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.331645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.331645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.332150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.332352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.332573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.332785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.332785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.333185 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.333403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.333403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.333794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.334011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.334011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.334382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.334612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.334872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.335084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.335084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.335478 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.335682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.335883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.336098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.336098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.336428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.336726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.336726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.337536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.337875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.337875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.338385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.338697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.338916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.339115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.339115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.339115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.339703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.339915 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.339915 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.340287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.340487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.340487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.340884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.341097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.341307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.341506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.341506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.341890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.342093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.342093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.342490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.342712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.342712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.343112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.343312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.343509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.343706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.343706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.344078 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.344287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.344287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.344671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.344890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.344890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.345263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.345474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.345678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.345874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.345874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.346260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.346456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.346456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.346890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.347101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.347101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.347487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.347698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.347912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.348109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.348109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.348501 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.348699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.348901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.349118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.349118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.349488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.349685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.349685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.350072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.350304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.350515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.350749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.350749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.351119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.351314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.351314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.351710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.351923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.351923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.352322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.352546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.352546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.352948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.353172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.353378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.353591 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.353591 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.353995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.354195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.354195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.354565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.354827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.354827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.355216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.355424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.355424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.355822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.356021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.356219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.356429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.356429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.356818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.357017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.357017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.357398 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.357607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.357607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.357980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.358192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.358192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.358628 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.358831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.359030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.359227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.359227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.359227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.external = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.359889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.360086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.360086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.360460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.connection = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.360671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.360671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.password = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.360671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] auth.token = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.360671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.360671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.361620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.361817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.361817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.362170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.362366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.362366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.363090 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.363283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.363283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.363657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.363846 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.363846 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.364240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.364436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.364436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.364799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.364989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.364989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.365366 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.365561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.365762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.365961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.365961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.366374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.366566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.366782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.366987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.366987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.367351 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.367558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.367558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.367962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.368167 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.368167 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.368564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.368784 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.368896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.369042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.369042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.369232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.369343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.369343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.369612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.369725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.369725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.369949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370124 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370658 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80225) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 08:09:45.370803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.370803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.371990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.371990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.371990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.371990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.371990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.371990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.371990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.371990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.371990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.373088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.373088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.373088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.373088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.373088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.373088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.373088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.373088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.373088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.374121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.374121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.374121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.374121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.374121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.374121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.374121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.374121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.374121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.375336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.375336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.375336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.375336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.375336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.375336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.375336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.375336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.375336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.376336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.376336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.376336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.376336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.376336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.376336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.376336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.376336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.376336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.377293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.377293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.377293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.377293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.377293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.377293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.377293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.377293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.378201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.378201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.378201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.378201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.378670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.378670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.378936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.379176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.379408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.379659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.379919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.380242 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.380482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.380719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.380966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.381214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.381451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.381708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.381938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.382183 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.382416 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.382716 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.382954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.383249 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.383536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.384384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.384624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.384867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.385119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.385381 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.385688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.386025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.386279 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.386514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.386788 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.387028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.387266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.387584 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] role.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.387883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] role.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.388122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.388369 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.388620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.388870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.389117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.389365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.389624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.389872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.390128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.390466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.390764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.391016 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.391261 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.391509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.392088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.392386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.392640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.392890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.393198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.393415 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.393704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.393967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.394216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.394544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.394801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.395037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.395268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.395558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.395813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.396062 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.396306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.396544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.396775 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.397009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.397280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.397553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.397800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.398056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.398302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.398537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.398809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.399054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.400061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.400472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.400741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.401070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.401227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.401462 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.401691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.401935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.402244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.402538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.connection = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.402809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.403057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.403295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.403543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.403853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.404102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.404337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.404651 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.404902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.405141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.405384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.405630 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.405965 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.406095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.406421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.406703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.406947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.407181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 08:09:45.407349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80226) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 08:09:45.413594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.413939 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.414304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.424278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.436933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.437175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.437382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.446602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.455094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.455509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.468165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.468584 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.519142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.609985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.650367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.650751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.671973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.672313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.727069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.727341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.727709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.727981 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.728286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.728511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.729469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.729786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.730143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.730494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.730829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.731208 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.758147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.758398 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.759243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.759433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.759433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.759767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.759967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.760109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.760249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.760399 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.760539 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.760676 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.760819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.760951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.772272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.772613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.786218 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.786465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.786698 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.786896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.787114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.787491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.787491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80225) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.788389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.788704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.788932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.789156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.789380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.789596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.789928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80226) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 08:09:45.807082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.807344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 08:09:45.807631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.807781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.807924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 08:09:45.808060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 08:09:45.808230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.808365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.808499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.808644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 08:09:45.808789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.808936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.809068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.809200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.809507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.809767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.820681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.820980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.821799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.822097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.822244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.822488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.822890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.823067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.830538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.831856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.833947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.834234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.834709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.835004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.835296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.835563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.835712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.835878 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.836021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.836200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.836341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.836478 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.836630 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.836767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.836903 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.837032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.837238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.837572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.837777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.838031 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.838310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.838615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.872830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.874750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.875042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.877103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.879044 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.879428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.881978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.882340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.882773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.886382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.886796 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.887173 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.891240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.891621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.895203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.895589 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.897010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.897383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.897740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.898102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.898454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.898838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.899243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.900098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.902435 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.902777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.903060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.903531 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.903882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.904197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.904471 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.904741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.920612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.920992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.924822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.925111 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.926779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.930166 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.931036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.931451 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.934058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.934355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.937060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.939715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.941100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.943665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.945258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.945652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.946593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.946989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.947364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.947736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.948279 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.948581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.948860 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.949139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.949417 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.949692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.968433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.968961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.969325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.969681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.969832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.970084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.970358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.970636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.978756 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.979568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.980761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.981136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.981434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.981724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.984598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.984868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:45.985022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.985200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.985420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.985570 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.985707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.985927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.986077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:45.986368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.001221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.001512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.001693 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.001852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.002054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.002205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.002348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.002534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.002708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.002909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.003087 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.003508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.003887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.004302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.022995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.026925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.028047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.030484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.031787 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.032189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.033827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.034243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.034540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.035715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.037511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.039460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.041245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.041245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.043010 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.044669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.048098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.048404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.048826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.048999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.049347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.051318 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.054675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.054977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.055268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.055940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.059018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.059276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.059629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.065788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.066092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.066389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.068052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80226) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.068409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.068760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.069138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.069503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.069837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.070192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.070536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.070930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.071801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.073524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.073876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.074229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.074629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80226) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.075424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80225) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 08:09:46.075727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.076017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.076299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.076626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.076923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.077205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.077484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.077913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.078234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.078533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.078842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.079375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.079525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80225) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 08:09:46.117933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x55c2269e99c0 pid: 80226 (default app) Jul 20 08:09:46.120712 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: WSGI app 0 (mountpoint='') ready in 8 seconds on interpreter 0x55c2269e99c0 pid: 80225 (default app) Jul 20 08:09:46.122223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e5e8c-4c2e-4b07-b5d1-6f19c38e9a3c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:09:46.122562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e5e8c-4c2e-4b07-b5d1-6f19c38e9a3c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:09:46.122961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e5e8c-4c2e-4b07-b5d1-6f19c38e9a3c None None] PATH_INFO: `/v3` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:09:46.127265 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1/1] 10.222.0.64 () {50 vars in 791 bytes} [Wed Jul 20 08:09:46 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:09:54.451401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36525b-c367-4c8e-b9fb-b76383e1b4d3 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:09:54.452218 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36525b-c367-4c8e-b9fb-b76383e1b4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:09:54.452366 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36525b-c367-4c8e-b9fb-b76383e1b4d3 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:09:54.456892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1/2] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:09:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:09:54.463423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f3333-0db0-41b8-9e99-02992e3234e0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:09:54.464202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f3333-0db0-41b8-9e99-02992e3234e0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:09:54.464340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f3333-0db0-41b8-9e99-02992e3234e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:09:54.544213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: INFO dbcounter [None req-9c9f3333-0db0-41b8-9e99-02992e3234e0 None None] Registered counter for database keystone Jul 20 08:09:54.569341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG oslo_db.sqlalchemy.engines [None req-9c9f3333-0db0-41b8-9e99-02992e3234e0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80226) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 08:09:54.581179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writer thread running {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 08:09:54.801643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG passlib.handlers.bcrypt [None req-9c9f3333-0db0-41b8-9e99-02992e3234e0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80226) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 08:09:54.802145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG passlib.handlers.bcrypt [None req-9c9f3333-0db0-41b8-9e99-02992e3234e0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80226) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 08:09:54.862922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9c9f3333-0db0-41b8-9e99-02992e3234e0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:09:54.864845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9c9f3333-0db0-41b8-9e99-02992e3234e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:09:55.044216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 2/3] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:09:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 582 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 08:09:55.281254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36525b-c367-4c8e-b9fb-b76383e1b4d3 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:09:55.367895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: INFO dbcounter [None req-be36525b-c367-4c8e-b9fb-b76383e1b4d3 None None] Registered counter for database keystone Jul 20 08:09:55.399677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG oslo_db.sqlalchemy.engines [None req-be36525b-c367-4c8e-b9fb-b76383e1b4d3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80225) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 08:09:55.412959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writer thread running {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 08:09:55.562332 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36525b-c367-4c8e-b9fb-b76383e1b4d3 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:09:55.567729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c39a4-dc7f-4383-8d83-2479a980c8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:09:55.568545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c39a4-dc7f-4383-8d83-2479a980c8bf admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:09:55.568922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c39a4-dc7f-4383-8d83-2479a980c8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:09:55.569290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c39a4-dc7f-4383-8d83-2479a980c8bf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:09:55.579840 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c39a4-dc7f-4383-8d83-2479a980c8bf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:09:55.635115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: WARNING keystone.common.rbac_enforcer.enforcer [None req-001c39a4-dc7f-4383-8d83-2479a980c8bf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 08:09:55.635584 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001c39a4-dc7f-4383-8d83-2479a980c8bf admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:09:55.647237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 2/4] 10.222.0.64 () {58 vars in 1324 bytes} [Wed Jul 20 08:09:55 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 08:09:55.654872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f3333-0db0-41b8-9e99-02992e3234e0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:09:55.671506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f3333-0db0-41b8-9e99-02992e3234e0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:09:55.676588 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e9509-cada-45af-ad6c-bdc0a1dc9633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:09:55.677760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e9509-cada-45af-ad6c-bdc0a1dc9633 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:09:55.678048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e9509-cada-45af-ad6c-bdc0a1dc9633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:09:55.678324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e9509-cada-45af-ad6c-bdc0a1dc9633 admin admin] PATH_INFO: `/v3/projects/10580c2140c64e24a4fb1ff11f27a985` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:09:55.680008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e9509-cada-45af-ad6c-bdc0a1dc9633 admin admin] RBAC: Authorizing `identity:get_project(project_id=10580c2140c64e24a4fb1ff11f27a985)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:09:55.736232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: WARNING keystone.common.rbac_enforcer.enforcer [None req-887e9509-cada-45af-ad6c-bdc0a1dc9633 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 08:09:55.736561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e9509-cada-45af-ad6c-bdc0a1dc9633 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:09:55.742868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 3/5] 10.222.0.64 () {56 vars in 1207 bytes} [Wed Jul 20 08:09:55 2022] GET /identity/v3/projects/10580c2140c64e24a4fb1ff11f27a985 => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:09:57.929493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1fa16-4ca3-4312-8812-3c6e03a5f632 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:09:57.930093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1fa16-4ca3-4312-8812-3c6e03a5f632 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:09:57.930481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1fa16-4ca3-4312-8812-3c6e03a5f632 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:09:57.939268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 3/6] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:09:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:09:57.946763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c208893-6422-4eb4-97fe-1118c65220fc None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:09:57.947383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c208893-6422-4eb4-97fe-1118c65220fc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:09:57.947383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c208893-6422-4eb4-97fe-1118c65220fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:09:57.969265 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5c208893-6422-4eb4-97fe-1118c65220fc None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:09:57.971056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5c208893-6422-4eb4-97fe-1118c65220fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:09:57.980724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 4/7] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:09:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 08:09:58.005256 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1fa16-4ca3-4312-8812-3c6e03a5f632 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:09:58.027006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1fa16-4ca3-4312-8812-3c6e03a5f632 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:09:58.031605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeca7ec-7b46-43d9-8ea3-371142a705fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:09:58.032319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeca7ec-7b46-43d9-8ea3-371142a705fb admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:09:58.032614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeca7ec-7b46-43d9-8ea3-371142a705fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:09:58.032896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeca7ec-7b46-43d9-8ea3-371142a705fb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:09:58.042134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeca7ec-7b46-43d9-8ea3-371142a705fb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:09:58.044110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeca7ec-7b46-43d9-8ea3-371142a705fb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:09:58.054754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 4/8] 10.222.0.64 () {58 vars in 1324 bytes} [Wed Jul 20 08:09:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 08:09:58.064135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c208893-6422-4eb4-97fe-1118c65220fc None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:09:58.081940 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c208893-6422-4eb4-97fe-1118c65220fc None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:09:58.087980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464aea6-bc2b-4f78-8ac0-b6fe37a26b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:09:58.088906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464aea6-bc2b-4f78-8ac0-b6fe37a26b6a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:09:58.089187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464aea6-bc2b-4f78-8ac0-b6fe37a26b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:09:58.089576 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464aea6-bc2b-4f78-8ac0-b6fe37a26b6a admin admin] PATH_INFO: `/v3/users/c5ec92316a2c452dab52aa49d7256f2f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:09:58.091258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464aea6-bc2b-4f78-8ac0-b6fe37a26b6a admin admin] RBAC: Authorizing `identity:get_user(user_id=c5ec92316a2c452dab52aa49d7256f2f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:09:58.093312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464aea6-bc2b-4f78-8ac0-b6fe37a26b6a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:09:58.095118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 5/9] 10.222.0.64 () {56 vars in 1198 bytes} [Wed Jul 20 08:09:58 2022] GET /identity/v3/users/c5ec92316a2c452dab52aa49d7256f2f => generated 247 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:00.473239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdad9d-5a40-4061-bf71-c725caf52d24 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:00.474045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdad9d-5a40-4061-bf71-c725caf52d24 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:00.474045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdad9d-5a40-4061-bf71-c725caf52d24 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:00.475635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 5/10] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:00.482407 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a87d5-1ba6-48d2-bdd6-d2a98c00d88e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:00.482998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a87d5-1ba6-48d2-bdd6-d2a98c00d88e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:00.483117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a87d5-1ba6-48d2-bdd6-d2a98c00d88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:00.503318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-479a87d5-1ba6-48d2-bdd6-d2a98c00d88e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:00.504648 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-479a87d5-1ba6-48d2-bdd6-d2a98c00d88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:00.647642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 6/11] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:00.673108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdad9d-5a40-4061-bf71-c725caf52d24 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:00.786225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1df474-7487-46e5-af49-39298ee39d7e None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:00.786515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1df474-7487-46e5-af49-39298ee39d7e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:00.787225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1df474-7487-46e5-af49-39298ee39d7e None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:00.788980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 7/12] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:10:00.795749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99b5d-dbf7-4dc8-8e55-c04cd5ada00f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:00.796152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99b5d-dbf7-4dc8-8e55-c04cd5ada00f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:00.796515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99b5d-dbf7-4dc8-8e55-c04cd5ada00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:00.817115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-44b99b5d-dbf7-4dc8-8e55-c04cd5ada00f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:00.818381 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-44b99b5d-dbf7-4dc8-8e55-c04cd5ada00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:00.839683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:00.922999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdad9d-5a40-4061-bf71-c725caf52d24 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:00.926763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 8/13] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:00.950693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b99b5d-dbf7-4dc8-8e55-c04cd5ada00f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:00.970357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3958f72-0010-459f-b004-08796e1633f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:00.971211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3958f72-0010-459f-b004-08796e1633f5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:00.971496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3958f72-0010-459f-b004-08796e1633f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:00.971774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3958f72-0010-459f-b004-08796e1633f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:00.987382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3958f72-0010-459f-b004-08796e1633f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-a3958f72-0010-459f-b004-08796e1633f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:01.000268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:01.001308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:01.002233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 6/14] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:10:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:01.008540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3958f72-0010-459f-b004-08796e1633f5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:01.064489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:01.097314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b99b5d-dbf7-4dc8-8e55-c04cd5ada00f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:01.142672 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eb473-6bee-48fa-836d-1c75fbc3da7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:01.143746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eb473-6bee-48fa-836d-1c75fbc3da7d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:01.144263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eb473-6bee-48fa-836d-1c75fbc3da7d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:01.144760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08eb473-6bee-48fa-836d-1c75fbc3da7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:01.158822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3958f72-0010-459f-b004-08796e1633f5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:01.185632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:01.201781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d068ef-a61f-4f1a-a23a-e901d20076a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:01.202670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d068ef-a61f-4f1a-a23a-e901d20076a3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:01.203062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d068ef-a61f-4f1a-a23a-e901d20076a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:01.203543 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d068ef-a61f-4f1a-a23a-e901d20076a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:01.204460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d068ef-a61f-4f1a-a23a-e901d20076a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:01.204880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d068ef-a61f-4f1a-a23a-e901d20076a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:01.206701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d068ef-a61f-4f1a-a23a-e901d20076a3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:01.223181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 7/15] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:01.230525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d068ef-a61f-4f1a-a23a-e901d20076a3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:01.253981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08eb473-6bee-48fa-836d-1c75fbc3da7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:01.256649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08eb473-6bee-48fa-836d-1c75fbc3da7d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:01.370120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:01.380060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d068ef-a61f-4f1a-a23a-e901d20076a3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:01.418610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 9/16] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:01.426841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eb473-6bee-48fa-836d-1c75fbc3da7d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:01.439126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfca23-fa6a-4ed1-88a4-760a46b82215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:01.440583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfca23-fa6a-4ed1-88a4-760a46b82215 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:01.441070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfca23-fa6a-4ed1-88a4-760a46b82215 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:01.441554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dfca23-fa6a-4ed1-88a4-760a46b82215 None admin] PATH_INFO: `/v3/users/c5ec92316a2c452dab52aa49d7256f2f` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:01.446489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfca23-fa6a-4ed1-88a4-760a46b82215 None admin] RBAC: Authorizing `identity:get_user(user_id=c5ec92316a2c452dab52aa49d7256f2f)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:01.449282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dfca23-fa6a-4ed1-88a4-760a46b82215 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:01.453743 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 8/17] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:01 2022] GET /identity/v3/users/c5ec92316a2c452dab52aa49d7256f2f => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:01.462434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfca23-fa6a-4ed1-88a4-760a46b82215 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:01.534709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:01.627502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08eb473-6bee-48fa-836d-1c75fbc3da7d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:01.648225 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dfca23-fa6a-4ed1-88a4-760a46b82215 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:01.687604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b945f07-5089-4d20-977b-31025404c1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:01.688802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b945f07-5089-4d20-977b-31025404c1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:01.689258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b945f07-5089-4d20-977b-31025404c1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:01.689636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b945f07-5089-4d20-977b-31025404c1a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:01.704973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b945f07-5089-4d20-977b-31025404c1a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:01.707312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b945f07-5089-4d20-977b-31025404c1a4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:01.722216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff12d9-e7a3-415b-b868-8619ede9361d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:01.723733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff12d9-e7a3-415b-b868-8619ede9361d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:01.724250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff12d9-e7a3-415b-b868-8619ede9361d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:01.724755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff12d9-e7a3-415b-b868-8619ede9361d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:01.727557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff12d9-e7a3-415b-b868-8619ede9361d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:01.730453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff12d9-e7a3-415b-b868-8619ede9361d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:01.733881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 9/18] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4b945f07-5089-4d20-977b-31025404c1a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:10:01.743527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:01.745105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:01.746661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:10:01.746661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:10:01.746661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:10:01.746661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:01.747066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff12d9-e7a3-415b-b868-8619ede9361d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:01.747489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 10/19] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:01.753222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b945f07-5089-4d20-977b-31025404c1a4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:01.786564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:01.904891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff12d9-e7a3-415b-b868-8619ede9361d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:01.924485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b945f07-5089-4d20-977b-31025404c1a4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:01.927545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:01.959060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacbddb-01e3-4cad-9bca-d59c8c834d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:01.959737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacbddb-01e3-4cad-9bca-d59c8c834d1d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:01.960032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacbddb-01e3-4cad-9bca-d59c8c834d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:01.960922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacbddb-01e3-4cad-9bca-d59c8c834d1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:01.961992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacbddb-01e3-4cad-9bca-d59c8c834d1d None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.domain.id=default, user.id=c5ec92316a2c452dab52aa49d7256f2f) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:01.962316 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacbddb-01e3-4cad-9bca-d59c8c834d1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:01.963901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacbddb-01e3-4cad-9bca-d59c8c834d1d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:01.969120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657aa5d4-d176-4561-aec2-85c13d6bb7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:01.969802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aa5d4-d176-4561-aec2-85c13d6bb7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:01.970095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aa5d4-d176-4561-aec2-85c13d6bb7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:01.970373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aa5d4-d176-4561-aec2-85c13d6bb7b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:01.971170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657aa5d4-d176-4561-aec2-85c13d6bb7b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:01.971483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657aa5d4-d176-4561-aec2-85c13d6bb7b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:01.973033 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657aa5d4-d176-4561-aec2-85c13d6bb7b5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:01.974369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 10/20] 10.222.0.64 () {56 vars in 1322 bytes} [Wed Jul 20 08:10:01 2022] GET /identity/v3/role_assignments?user.id=c5ec92316a2c452dab52aa49d7256f2f&scope.domain.id=default&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 233 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:01.998941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 11/21] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:05.561737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7055-1fa8-43ac-99aa-1660077f933b None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:05.562831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7055-1fa8-43ac-99aa-1660077f933b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:05.563360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7055-1fa8-43ac-99aa-1660077f933b None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:05.565142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 11/22] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:05.577361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ca209-1ef8-4829-8a6b-2f9e02701a64 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:05.578477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ca209-1ef8-4829-8a6b-2f9e02701a64 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:05.579042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ca209-1ef8-4829-8a6b-2f9e02701a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:05.603022 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ba5ca209-1ef8-4829-8a6b-2f9e02701a64 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:05.604734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ba5ca209-1ef8-4829-8a6b-2f9e02701a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:05.742948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 12/23] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:05.813941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d7055-1fa8-43ac-99aa-1660077f933b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:05.904100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42431b-95de-40fc-bec3-5598ec881ee5 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:05.905071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42431b-95de-40fc-bec3-5598ec881ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:05.905914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42431b-95de-40fc-bec3-5598ec881ee5 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:05.908896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 13/24] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:10:05.918334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20702-9153-4a13-a760-32a93a2110df None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:05.919290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20702-9153-4a13-a760-32a93a2110df None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:05.920072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20702-9153-4a13-a760-32a93a2110df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:05.978150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-04d20702-9153-4a13-a760-32a93a2110df None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:05.981147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-04d20702-9153-4a13-a760-32a93a2110df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:06.143362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:06.157847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d7055-1fa8-43ac-99aa-1660077f933b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:06.196449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 14/25] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:06.216301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330afcc-707c-48b0-a652-a202d3b9dd3e None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:06.216693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330afcc-707c-48b0-a652-a202d3b9dd3e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:06.216939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330afcc-707c-48b0-a652-a202d3b9dd3e None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:06.218450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 15/26] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:06.230921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ba94-3f73-4651-9e1f-79dca4161ed7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:06.231484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ba94-3f73-4651-9e1f-79dca4161ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:06.231962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55ba94-3f73-4651-9e1f-79dca4161ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:06.261203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ec55ba94-3f73-4651-9e1f-79dca4161ed7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:06.263207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ec55ba94-3f73-4651-9e1f-79dca4161ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:06.269138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd881cf-765a-40cf-ab86-06b5ff9db302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:06.270339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd881cf-765a-40cf-ab86-06b5ff9db302 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:06.270823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd881cf-765a-40cf-ab86-06b5ff9db302 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:06.271270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd881cf-765a-40cf-ab86-06b5ff9db302 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:06.297937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd881cf-765a-40cf-ab86-06b5ff9db302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-6dd881cf-765a-40cf-ab86-06b5ff9db302 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:06.306912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:06.307972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:06.308732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:06.320938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 12/27] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:10:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 508 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:06.326292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd881cf-765a-40cf-ab86-06b5ff9db302 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:06.332438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:06.476378 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 16/28] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:06.477606 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55ba94-3f73-4651-9e1f-79dca4161ed7 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:06.603520 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55ba94-3f73-4651-9e1f-79dca4161ed7 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:06.644663 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd881cf-765a-40cf-ab86-06b5ff9db302 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:06.664114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0827ab-6ebd-4ed8-a051-c40da605bc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:06.666103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0827ab-6ebd-4ed8-a051-c40da605bc60 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:06.666434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0827ab-6ebd-4ed8-a051-c40da605bc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:06.666714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0827ab-6ebd-4ed8-a051-c40da605bc60 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:06.667611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0827ab-6ebd-4ed8-a051-c40da605bc60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:06.668413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0827ab-6ebd-4ed8-a051-c40da605bc60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:06.670113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0827ab-6ebd-4ed8-a051-c40da605bc60 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:06.692892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 17/29] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:06.695699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc69b07-34f0-456a-beca-15f45903f48e None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:06.696198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc69b07-34f0-456a-beca-15f45903f48e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:06.696702 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc69b07-34f0-456a-beca-15f45903f48e None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:06.699727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 18/30] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:06.701034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc69b07-34f0-456a-beca-15f45903f48e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:06.729061 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f582b19b-cdaf-41ac-b54d-8f668674aa09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:06.730256 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582b19b-cdaf-41ac-b54d-8f668674aa09 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:06.730814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582b19b-cdaf-41ac-b54d-8f668674aa09 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:06.731536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f582b19b-cdaf-41ac-b54d-8f668674aa09 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:06.733082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582b19b-cdaf-41ac-b54d-8f668674aa09 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:06.735860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f582b19b-cdaf-41ac-b54d-8f668674aa09 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:06.742765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:06.919527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc69b07-34f0-456a-beca-15f45903f48e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:06.945034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 13/31] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Jul 20 08:10:06 2022] POST /identity/v3/roles => generated 225 bytes in 615 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:06.945034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c1316-7864-4bdb-9f4a-77254888cd20 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:06.945034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c1316-7864-4bdb-9f4a-77254888cd20 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:06.945519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c1316-7864-4bdb-9f4a-77254888cd20 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:06.948062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 14/32] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:06.949491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c1316-7864-4bdb-9f4a-77254888cd20 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:06.986263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:07.001938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bd3da-9737-42af-af9b-b1fd6e023d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:07.003849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bd3da-9737-42af-af9b-b1fd6e023d17 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:07.004465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bd3da-9737-42af-af9b-b1fd6e023d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:07.005099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bd3da-9737-42af-af9b-b1fd6e023d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:07.008073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bd3da-9737-42af-af9b-b1fd6e023d17 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:07.011152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bd3da-9737-42af-af9b-b1fd6e023d17 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:07.182937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312c1316-7864-4bdb-9f4a-77254888cd20 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:07.250880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 19/33] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Jul 20 08:10:06 2022] POST /identity/v3/roles => generated 219 bytes in 550 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:07.253891 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dd237-f38e-4b54-afdd-4a1a2e533904 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:07.254391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dd237-f38e-4b54-afdd-4a1a2e533904 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:07.255164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dd237-f38e-4b54-afdd-4a1a2e533904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:07.270003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac8c1d-25fa-4b4b-86a1-333ec019c908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:07.271993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac8c1d-25fa-4b4b-86a1-333ec019c908 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:07.273054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac8c1d-25fa-4b4b-86a1-333ec019c908 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:07.273572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac8c1d-25fa-4b4b-86a1-333ec019c908 None admin] PATH_INFO: `/v3/users/c5ec92316a2c452dab52aa49d7256f2f` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:07.276538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac8c1d-25fa-4b4b-86a1-333ec019c908 None admin] RBAC: Authorizing `identity:get_user(user_id=c5ec92316a2c452dab52aa49d7256f2f)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:07.279732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac8c1d-25fa-4b4b-86a1-333ec019c908 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:07.283368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 15/34] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:06 2022] GET /identity/v3/users/c5ec92316a2c452dab52aa49d7256f2f => generated 247 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:07.284983 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e41dd237-f38e-4b54-afdd-4a1a2e533904 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:07.286360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe2fca-6d18-4560-a6bb-305deddaf76b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:07.286944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e41dd237-f38e-4b54-afdd-4a1a2e533904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:07.287448 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe2fca-6d18-4560-a6bb-305deddaf76b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:07.288790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe2fca-6d18-4560-a6bb-305deddaf76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:07.318377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG passlib.handlers.bcrypt [None req-02fe2fca-6d18-4560-a6bb-305deddaf76b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80225) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 08:10:07.320102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG passlib.handlers.bcrypt [None req-02fe2fca-6d18-4560-a6bb-305deddaf76b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80225) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 08:10:07.375090 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-02fe2fca-6d18-4560-a6bb-305deddaf76b None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:07.378023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-02fe2fca-6d18-4560-a6bb-305deddaf76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:07.395809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 20/35] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:07.397595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2bdeb-a9c0-4d78-ad5f-d7e41dd26b1d None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:07.398098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2bdeb-a9c0-4d78-ad5f-d7e41dd26b1d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:07.398515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2bdeb-a9c0-4d78-ad5f-d7e41dd26b1d None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:07.400283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 21/36] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:07.401290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2bdeb-a9c0-4d78-ad5f-d7e41dd26b1d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:07.420398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:07.538622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 16/37] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:07.540160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bf2e9-0e6a-46b8-82e0-0f04ea23ae4c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:07.540465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bf2e9-0e6a-46b8-82e0-0f04ea23ae4c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:07.540748 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bf2e9-0e6a-46b8-82e0-0f04ea23ae4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:07.565108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ad5bf2e9-0e6a-46b8-82e0-0f04ea23ae4c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:07.567933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ad5bf2e9-0e6a-46b8-82e0-0f04ea23ae4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:07.587860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:07.596133 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2bdeb-a9c0-4d78-ad5f-d7e41dd26b1d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:07.643934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cd25f-6f54-43e2-ba39-545e71a9a989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:07.644762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd25f-6f54-43e2-ba39-545e71a9a989 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:07.645054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd25f-6f54-43e2-ba39-545e71a9a989 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:07.645337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd25f-6f54-43e2-ba39-545e71a9a989 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:07.646826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cd25f-6f54-43e2-ba39-545e71a9a989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:07.648555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55cd25f-6f54-43e2-ba39-545e71a9a989 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:07.650384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 22/38] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:07.651834 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cd25f-6f54-43e2-ba39-545e71a9a989 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:07.697356 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 17/39] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:07.698433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bf2e9-0e6a-46b8-82e0-0f04ea23ae4c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:07.774257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:07.804794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cd25f-6f54-43e2-ba39-545e71a9a989 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:07.838294 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5bf2e9-0e6a-46b8-82e0-0f04ea23ae4c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:07.851832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2422e8-f221-4be8-b1ba-749815505455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:07.853102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2422e8-f221-4be8-b1ba-749815505455 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:07.853463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2422e8-f221-4be8-b1ba-749815505455 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:07.853892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2422e8-f221-4be8-b1ba-749815505455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:07.874922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2422e8-f221-4be8-b1ba-749815505455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:07.877096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2422e8-f221-4be8-b1ba-749815505455 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:07.893711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6074753-a219-4ad4-86f9-439335e1251c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:07.894929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6074753-a219-4ad4-86f9-439335e1251c None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:07.895452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6074753-a219-4ad4-86f9-439335e1251c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:07.895959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6074753-a219-4ad4-86f9-439335e1251c None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0c2422e8-f221-4be8-b1ba-749815505455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:10:07.896689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:07.898345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:07.899467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:10:07.899467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:10:07.899467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:10:07.899467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:07.899665 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6074753-a219-4ad4-86f9-439335e1251c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:07.899730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 23/40] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:07.899730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2422e8-f221-4be8-b1ba-749815505455 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:07.900788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6074753-a219-4ad4-86f9-439335e1251c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:07.921243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 18/41] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Jul 20 08:10:07 2022] POST /identity/v3/roles => generated 223 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:07.922540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6074753-a219-4ad4-86f9-439335e1251c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:07.947446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:08.090974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6074753-a219-4ad4-86f9-439335e1251c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:08.140250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d24549-5c49-4f64-aada-10f7116c0837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:08.141093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d24549-5c49-4f64-aada-10f7116c0837 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:08.141388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d24549-5c49-4f64-aada-10f7116c0837 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:08.141673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d24549-5c49-4f64-aada-10f7116c0837 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:08.143393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d24549-5c49-4f64-aada-10f7116c0837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:08.144887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d24549-5c49-4f64-aada-10f7116c0837 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:08.146847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 19/42] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:08.151635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d24549-5c49-4f64-aada-10f7116c0837 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:08.166123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2422e8-f221-4be8-b1ba-749815505455 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:08.208219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:08.220924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137e19f-2fe2-4e5f-8773-479f984ea9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:08.221932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137e19f-2fe2-4e5f-8773-479f984ea9f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:08.222279 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137e19f-2fe2-4e5f-8773-479f984ea9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:08.222649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137e19f-2fe2-4e5f-8773-479f984ea9f1 None admin] PATH_INFO: `/v3/domains/default/users/c5ec92316a2c452dab52aa49d7256f2f/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:08.225973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137e19f-2fe2-4e5f-8773-479f984ea9f1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c5ec92316a2c452dab52aa49d7256f2f, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:08.228166 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137e19f-2fe2-4e5f-8773-479f984ea9f1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:08.249396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 24/43] 10.222.0.64 () {58 vars in 1361 bytes} [Wed Jul 20 08:10:07 2022] PUT /identity/v3/domains/default/users/c5ec92316a2c452dab52aa49d7256f2f/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:10:08.251998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137e19f-2fe2-4e5f-8773-479f984ea9f1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:08.252657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:08.297668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d24549-5c49-4f64-aada-10f7116c0837 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:08.348731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c690604-b9bf-45fc-bae6-de9db2e43243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:08.349356 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c690604-b9bf-45fc-bae6-de9db2e43243 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:08.349652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c690604-b9bf-45fc-bae6-de9db2e43243 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:08.349940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c690604-b9bf-45fc-bae6-de9db2e43243 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:08.351405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c690604-b9bf-45fc-bae6-de9db2e43243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:08.351843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c690604-b9bf-45fc-bae6-de9db2e43243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:08.353567 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c690604-b9bf-45fc-bae6-de9db2e43243 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:08.379232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 20/44] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:08.386695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c690604-b9bf-45fc-bae6-de9db2e43243 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:08.465782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137e19f-2fe2-4e5f-8773-479f984ea9f1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:08.517709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f25eb8-ca8c-4f58-b1ee-b5f90dc871c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:08.518785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f25eb8-ca8c-4f58-b1ee-b5f90dc871c0 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:08.519224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f25eb8-ca8c-4f58-b1ee-b5f90dc871c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:08.519664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f25eb8-ca8c-4f58-b1ee-b5f90dc871c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:08.520838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f25eb8-ca8c-4f58-b1ee-b5f90dc871c0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:08.525260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f25eb8-ca8c-4f58-b1ee-b5f90dc871c0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:08.539961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c690604-b9bf-45fc-bae6-de9db2e43243 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:08.575610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 25/45] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:10:08 2022] POST /identity/v3/projects => generated 312 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:08.583822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:08.598949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4cee17-c656-4709-93ed-d0fcef9536c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:08.599642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cee17-c656-4709-93ed-d0fcef9536c9 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:08.599934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cee17-c656-4709-93ed-d0fcef9536c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:08.600216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4cee17-c656-4709-93ed-d0fcef9536c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:08.601068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cee17-c656-4709-93ed-d0fcef9536c9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:08.602632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4cee17-c656-4709-93ed-d0fcef9536c9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:08.650634 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 21/46] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:10:08 2022] POST /identity/v3/projects => generated 301 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:10.634332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62972b8-159a-4816-9ce3-3f2362fb49e5 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:10.634887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62972b8-159a-4816-9ce3-3f2362fb49e5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:10.635337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62972b8-159a-4816-9ce3-3f2362fb49e5 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:10.638026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 26/47] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:10.648032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339f5d1-ab21-4549-8047-922cae5dc58a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:10.648788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339f5d1-ab21-4549-8047-922cae5dc58a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:10.648788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339f5d1-ab21-4549-8047-922cae5dc58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:10.670685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6339f5d1-ab21-4549-8047-922cae5dc58a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:10.672191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6339f5d1-ab21-4549-8047-922cae5dc58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:10.771089 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 22/48] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:10.797542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62972b8-159a-4816-9ce3-3f2362fb49e5 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:10.851655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:10.910216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34386511-3f22-4ef0-aaa5-f5611fcdd96d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:10.910707 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34386511-3f22-4ef0-aaa5-f5611fcdd96d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:10.911214 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34386511-3f22-4ef0-aaa5-f5611fcdd96d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:10.914208 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 23/49] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:10:10.922860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d7e32-ba01-405e-b856-32a863b5ddf7 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:10.923280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d7e32-ba01-405e-b856-32a863b5ddf7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:10.923700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d7e32-ba01-405e-b856-32a863b5ddf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:10.936291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62972b8-159a-4816-9ce3-3f2362fb49e5 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:10.980549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9793ed1-a645-429b-9803-ca0803493cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:10.981478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9793ed1-a645-429b-9803-ca0803493cbb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:10.981832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9793ed1-a645-429b-9803-ca0803493cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:10.982176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9793ed1-a645-429b-9803-ca0803493cbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:10.999041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9793ed1-a645-429b-9803-ca0803493cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d9793ed1-a645-429b-9803-ca0803493cbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:11.009705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:11.011764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:11.013198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 27/50] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:10:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:11.019687 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9793ed1-a645-429b-9803-ca0803493cbb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:11.056250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-928d7e32-ba01-405e-b856-32a863b5ddf7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:11.057723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-928d7e32-ba01-405e-b856-32a863b5ddf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:11.151687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 24/51] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:11.161368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:11.166445 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9793ed1-a645-429b-9803-ca0803493cbb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:11.177006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d7e32-ba01-405e-b856-32a863b5ddf7 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:11.213352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66100a80-b3ae-4d0a-b723-b8356f857bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:11.214216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66100a80-b3ae-4d0a-b723-b8356f857bec None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:11.214712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66100a80-b3ae-4d0a-b723-b8356f857bec None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:11.215005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66100a80-b3ae-4d0a-b723-b8356f857bec None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:11.215788 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66100a80-b3ae-4d0a-b723-b8356f857bec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:11.216105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66100a80-b3ae-4d0a-b723-b8356f857bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:11.217603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66100a80-b3ae-4d0a-b723-b8356f857bec None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:11.228738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 28/52] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:11.234638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66100a80-b3ae-4d0a-b723-b8356f857bec None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:11.235159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:11.308321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d7e32-ba01-405e-b856-32a863b5ddf7 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:11.353001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440fa2f-7f9e-49f1-a39a-6e8fbb37fd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:11.354617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440fa2f-7f9e-49f1-a39a-6e8fbb37fd8b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:11.355157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440fa2f-7f9e-49f1-a39a-6e8fbb37fd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:11.355761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440fa2f-7f9e-49f1-a39a-6e8fbb37fd8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:11.357964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440fa2f-7f9e-49f1-a39a-6e8fbb37fd8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:11.360905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440fa2f-7f9e-49f1-a39a-6e8fbb37fd8b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:11.366641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 25/53] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:11.371213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440fa2f-7f9e-49f1-a39a-6e8fbb37fd8b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:11.398869 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66100a80-b3ae-4d0a-b723-b8356f857bec None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:11.441478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f165d-ca6e-4d6f-9a4f-1b04287ae8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:11.442330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f165d-ca6e-4d6f-9a4f-1b04287ae8de None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:11.442644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f165d-ca6e-4d6f-9a4f-1b04287ae8de None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:11.442968 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f165d-ca6e-4d6f-9a4f-1b04287ae8de None admin] PATH_INFO: `/v3/users/c5ec92316a2c452dab52aa49d7256f2f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:11.445044 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f165d-ca6e-4d6f-9a4f-1b04287ae8de None admin] RBAC: Authorizing `identity:get_user(user_id=c5ec92316a2c452dab52aa49d7256f2f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:11.446791 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f165d-ca6e-4d6f-9a4f-1b04287ae8de None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:11.448713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 29/54] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:11 2022] GET /identity/v3/users/c5ec92316a2c452dab52aa49d7256f2f => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:11.456235 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f165d-ca6e-4d6f-9a4f-1b04287ae8de None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:11.495681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:11.515684 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440fa2f-7f9e-49f1-a39a-6e8fbb37fd8b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:11.542300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:11.571541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0f0b2-be34-46a3-896e-7705c69dd15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:11.572139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0f0b2-be34-46a3-896e-7705c69dd15b None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:11.572658 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0f0b2-be34-46a3-896e-7705c69dd15b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:11.573137 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0f0b2-be34-46a3-896e-7705c69dd15b None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:11.574447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0f0b2-be34-46a3-896e-7705c69dd15b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:11.577200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0f0b2-be34-46a3-896e-7705c69dd15b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:11.609149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f165d-ca6e-4d6f-9a4f-1b04287ae8de None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:11.616975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 26/55] 10.222.0.64 () {60 vars in 1140 bytes} [Wed Jul 20 08:10:11 2022] POST /identity/v3/projects => generated 298 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:10:11.677211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90c3d3-8d04-4728-8842-05c2c708a50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:11.679032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90c3d3-8d04-4728-8842-05c2c708a50e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:11.679419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90c3d3-8d04-4728-8842-05c2c708a50e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:11.679787 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90c3d3-8d04-4728-8842-05c2c708a50e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:11.681488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90c3d3-8d04-4728-8842-05c2c708a50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:11.683410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90c3d3-8d04-4728-8842-05c2c708a50e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:11.685546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 30/56] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:11.692760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0f0b2-be34-46a3-896e-7705c69dd15b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:11.862534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0f0b2-be34-46a3-896e-7705c69dd15b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:11.923514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652651a-9e01-4f8e-b426-89c3045e52a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:11.924578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652651a-9e01-4f8e-b426-89c3045e52a8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:11.925086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652651a-9e01-4f8e-b426-89c3045e52a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:11.925571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652651a-9e01-4f8e-b426-89c3045e52a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:11.927279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652651a-9e01-4f8e-b426-89c3045e52a8 None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.domain.id=default, user.id=c5ec92316a2c452dab52aa49d7256f2f) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:11.927847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652651a-9e01-4f8e-b426-89c3045e52a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:11.931179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652651a-9e01-4f8e-b426-89c3045e52a8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:11.950702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 27/57] 10.222.0.64 () {56 vars in 1322 bytes} [Wed Jul 20 08:10:11 2022] GET /identity/v3/role_assignments?user.id=c5ec92316a2c452dab52aa49d7256f2f&scope.domain.id=default&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 530 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:13.814429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbc92-a5c5-402a-91c2-4dcd99a078d8 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:13.815083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbc92-a5c5-402a-91c2-4dcd99a078d8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:13.815083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fbc92-a5c5-402a-91c2-4dcd99a078d8 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:13.816457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 31/58] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:13.822420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f4dc8-9326-46aa-82f9-f3a14edd2f04 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:13.822947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f4dc8-9326-46aa-82f9-f3a14edd2f04 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:13.823067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f4dc8-9326-46aa-82f9-f3a14edd2f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:13.844681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6e1f4dc8-9326-46aa-82f9-f3a14edd2f04 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:13.845948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6e1f4dc8-9326-46aa-82f9-f3a14edd2f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:13.874076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:13.924498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 28/59] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:13.950831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fbc92-a5c5-402a-91c2-4dcd99a078d8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:14.059890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:14.068207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fbc92-a5c5-402a-91c2-4dcd99a078d8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:14.107300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf71c6-5003-4ae4-978c-7216f882ae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:14.108119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf71c6-5003-4ae4-978c-7216f882ae8d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:14.108411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf71c6-5003-4ae4-978c-7216f882ae8d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:14.108697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf71c6-5003-4ae4-978c-7216f882ae8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:14.110052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf71c6-5003-4ae4-978c-7216f882ae8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:14.111858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf71c6-5003-4ae4-978c-7216f882ae8d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:14.113571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 32/60] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:14.119654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f4dc8-9326-46aa-82f9-f3a14edd2f04 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:14.239201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f4dc8-9326-46aa-82f9-f3a14edd2f04 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:14.284509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc975e3c-9922-411f-b585-792bb570650d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:14.285851 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc975e3c-9922-411f-b585-792bb570650d None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:14.286157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc975e3c-9922-411f-b585-792bb570650d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:14.286466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc975e3c-9922-411f-b585-792bb570650d None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:14.288064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc975e3c-9922-411f-b585-792bb570650d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:14.291398 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc975e3c-9922-411f-b585-792bb570650d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:14.321657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 29/61] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:10:14 2022] POST /identity/v3/users => generated 275 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:18.702270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87e7cb-0a0d-41ee-aaaf-7c040ea3dd68 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:18.703154 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87e7cb-0a0d-41ee-aaaf-7c040ea3dd68 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:18.703154 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87e7cb-0a0d-41ee-aaaf-7c040ea3dd68 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:18.704707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 33/62] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:10:18.714254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ae3f0-aa30-46b5-a5f9-6ab642548c02 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:18.715805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ae3f0-aa30-46b5-a5f9-6ab642548c02 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:18.716597 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ae3f0-aa30-46b5-a5f9-6ab642548c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:18.742874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce124cfd-9315-4dc2-87a7-3778ad1cdb70 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:18.743544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce124cfd-9315-4dc2-87a7-3778ad1cdb70 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:18.744051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce124cfd-9315-4dc2-87a7-3778ad1cdb70 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:18.746129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 34/63] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:18.752253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e47da-22d1-4366-8212-7cac0912af83 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:18.752971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e47da-22d1-4366-8212-7cac0912af83 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:18.753470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e47da-22d1-4366-8212-7cac0912af83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:18.759656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-545ae3f0-aa30-46b5-a5f9-6ab642548c02 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:18.761746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-545ae3f0-aa30-46b5-a5f9-6ab642548c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:18.803819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:18.809769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-758e47da-22d1-4366-8212-7cac0912af83 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:18.812024 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-758e47da-22d1-4366-8212-7cac0912af83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:19.008511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 35/64] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:19.016122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 30/65] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:19.039798 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e47da-22d1-4366-8212-7cac0912af83 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:19.059826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce349c15-e557-436d-be76-738107e9d329 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:19.060339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce349c15-e557-436d-be76-738107e9d329 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:19.060798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce349c15-e557-436d-be76-738107e9d329 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:19.065540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 31/66] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:19.067378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce349c15-e557-436d-be76-738107e9d329 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:19.237102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:19.259946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e47da-22d1-4366-8212-7cac0912af83 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:19.306687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce349c15-e557-436d-be76-738107e9d329 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:19.346805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710495d-a067-4fa2-bbb6-a342ae19cea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:19.348259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710495d-a067-4fa2-bbb6-a342ae19cea2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:19.348722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710495d-a067-4fa2-bbb6-a342ae19cea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:19.349215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710495d-a067-4fa2-bbb6-a342ae19cea2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:19.366573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d120077-17d2-4db7-a7a1-a2e3183e1735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:19.367967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d120077-17d2-4db7-a7a1-a2e3183e1735 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:19.368668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d120077-17d2-4db7-a7a1-a2e3183e1735 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:19.368829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d120077-17d2-4db7-a7a1-a2e3183e1735 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:19.371481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710495d-a067-4fa2-bbb6-a342ae19cea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4710495d-a067-4fa2-bbb6-a342ae19cea2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:19.387236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:19.388178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:19.388929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:19.389505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 36/67] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:19.391800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9612097-0589-4a1a-8c15-26efd3da0223 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:19.392295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9612097-0589-4a1a-8c15-26efd3da0223 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:19.392789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9612097-0589-4a1a-8c15-26efd3da0223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:19.398059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d120077-17d2-4db7-a7a1-a2e3183e1735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-2d120077-17d2-4db7-a7a1-a2e3183e1735 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:19.415114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:19.416093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:19.417151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:19.424778 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c9612097-0589-4a1a-8c15-26efd3da0223 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:19.424778 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c9612097-0589-4a1a-8c15-26efd3da0223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:19.425074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 32/68] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:10:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:19.425074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b66c7-c483-4c27-be73-46010ed667e2 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:19.425074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b66c7-c483-4c27-be73-46010ed667e2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:19.425074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b66c7-c483-4c27-be73-46010ed667e2 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:19.425074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 33/69] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:19.426659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b66c7-c483-4c27-be73-46010ed667e2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:19.433857 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:19.538955 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 37/70] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:19.540728 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9612097-0589-4a1a-8c15-26efd3da0223 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:19.599983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b66c7-c483-4c27-be73-46010ed667e2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:19.647201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906bb0e-d0fc-4b59-bd51-8dc7eb26d82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:19.648299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906bb0e-d0fc-4b59-bd51-8dc7eb26d82a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:19.648771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906bb0e-d0fc-4b59-bd51-8dc7eb26d82a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:19.649234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906bb0e-d0fc-4b59-bd51-8dc7eb26d82a None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:19.650538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906bb0e-d0fc-4b59-bd51-8dc7eb26d82a None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:19.651090 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906bb0e-d0fc-4b59-bd51-8dc7eb26d82a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:19.653553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906bb0e-d0fc-4b59-bd51-8dc7eb26d82a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:19.667675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 34/71] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:19.669826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89304a-ca67-4b1c-9cdc-b9f294608ecb None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:19.670310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89304a-ca67-4b1c-9cdc-b9f294608ecb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:19.670826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb89304a-ca67-4b1c-9cdc-b9f294608ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:19.673104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:19.694272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cb89304a-ca67-4b1c-9cdc-b9f294608ecb None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:19.695571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cb89304a-ca67-4b1c-9cdc-b9f294608ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:19.697258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9612097-0589-4a1a-8c15-26efd3da0223 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:19.749794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef14f-8506-4eea-bd77-1907e906d55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:19.750548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef14f-8506-4eea-bd77-1907e906d55d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:19.750789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef14f-8506-4eea-bd77-1907e906d55d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:19.751072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef14f-8506-4eea-bd77-1907e906d55d None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:19.751848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef14f-8506-4eea-bd77-1907e906d55d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:19.752168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef14f-8506-4eea-bd77-1907e906d55d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:19.753618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef14f-8506-4eea-bd77-1907e906d55d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:19.764400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 38/72] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:10:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:19.766403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290073c-7502-4ea4-9c9a-ddc73658404b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:19.766951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290073c-7502-4ea4-9c9a-ddc73658404b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:19.767424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290073c-7502-4ea4-9c9a-ddc73658404b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:19.769256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 39/73] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:19.770141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290073c-7502-4ea4-9c9a-ddc73658404b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:19.778525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:19.788848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 35/74] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:19.789810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89304a-ca67-4b1c-9cdc-b9f294608ecb None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:19.909189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb89304a-ca67-4b1c-9cdc-b9f294608ecb None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:19.926913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290073c-7502-4ea4-9c9a-ddc73658404b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:19.954953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b481657-e2b2-4a47-9f8e-c904e31d586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:19.956003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b481657-e2b2-4a47-9f8e-c904e31d586d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:19.956497 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b481657-e2b2-4a47-9f8e-c904e31d586d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:19.956903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b481657-e2b2-4a47-9f8e-c904e31d586d None admin] PATH_INFO: `/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:19.968175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14307fd-c05b-461c-8b36-5ed13e323f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:19.969212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14307fd-c05b-461c-8b36-5ed13e323f0e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:19.969584 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14307fd-c05b-461c-8b36-5ed13e323f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:19.969946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14307fd-c05b-461c-8b36-5ed13e323f0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:19.977527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b481657-e2b2-4a47-9f8e-c904e31d586d None admin] RBAC: Authorizing `identity:get_user(user_id=48574d0c8c5d4648b0a8c6d7d6d34287)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:19.979074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b481657-e2b2-4a47-9f8e-c904e31d586d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:19.980887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 36/75] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:19 2022] GET /identity/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:19.981692 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b481657-e2b2-4a47-9f8e-c904e31d586d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:19.988171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14307fd-c05b-461c-8b36-5ed13e323f0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d14307fd-c05b-461c-8b36-5ed13e323f0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:19.999599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:20.000765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:20.001818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 40/76] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:10:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:20.002980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123cfa1-1855-4336-aa83-abfb4f978ff9 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:20.003497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123cfa1-1855-4336-aa83-abfb4f978ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:20.004001 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123cfa1-1855-4336-aa83-abfb4f978ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:20.026552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:20.034346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7123cfa1-1855-4336-aa83-abfb4f978ff9 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:20.035636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7123cfa1-1855-4336-aa83-abfb4f978ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:20.068279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:20.121930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b481657-e2b2-4a47-9f8e-c904e31d586d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:20.136158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 41/77] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:20.137174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123cfa1-1855-4336-aa83-abfb4f978ff9 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:20.166499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7249ba4-fa00-4cf3-b326-ed6995801381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:20.167704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7249ba4-fa00-4cf3-b326-ed6995801381 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:20.168152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7249ba4-fa00-4cf3-b326-ed6995801381 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:20.168526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7249ba4-fa00-4cf3-b326-ed6995801381 None admin] PATH_INFO: `/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:20.169971 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7249ba4-fa00-4cf3-b326-ed6995801381 None admin] RBAC: Authorizing `identity:get_user(user_id=48574d0c8c5d4648b0a8c6d7d6d34287)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:20.171932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7249ba4-fa00-4cf3-b326-ed6995801381 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:20.173802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 37/78] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:19 2022] GET /identity/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:20.174413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7249ba4-fa00-4cf3-b326-ed6995801381 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:20.268138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123cfa1-1855-4336-aa83-abfb4f978ff9 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:20.305905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7249ba4-fa00-4cf3-b326-ed6995801381 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:20.313845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02ae25-a0ba-47d5-b366-0b0ed0f0270f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:20.313845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02ae25-a0ba-47d5-b366-0b0ed0f0270f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:20.314142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02ae25-a0ba-47d5-b366-0b0ed0f0270f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:20.315030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02ae25-a0ba-47d5-b366-0b0ed0f0270f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:20.315711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02ae25-a0ba-47d5-b366-0b0ed0f0270f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:20.317211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02ae25-a0ba-47d5-b366-0b0ed0f0270f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:20.319727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 42/79] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:20.320482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02ae25-a0ba-47d5-b366-0b0ed0f0270f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:20.325722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:20.335626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:20.358837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c698ad-851d-419c-82a3-44835df2b633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:20.359521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c698ad-851d-419c-82a3-44835df2b633 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:20.359812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c698ad-851d-419c-82a3-44835df2b633 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:20.360098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c698ad-851d-419c-82a3-44835df2b633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:20.438752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c698ad-851d-419c-82a3-44835df2b633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:20.440390 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c698ad-851d-419c-82a3-44835df2b633 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:20.446446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02ae25-a0ba-47d5-b366-0b0ed0f0270f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:20.489692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0f73a-9604-48ee-9a75-81fd480f36a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:20.491101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0f73a-9604-48ee-9a75-81fd480f36a8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:20.491609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0f73a-9604-48ee-9a75-81fd480f36a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:20.491968 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0f73a-9604-48ee-9a75-81fd480f36a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:20.492735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0f73a-9604-48ee-9a75-81fd480f36a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:20.493043 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0f73a-9604-48ee-9a75-81fd480f36a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:20.494479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0f73a-9604-48ee-9a75-81fd480f36a8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:20.503267 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 43/80] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:20.509871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0f73a-9604-48ee-9a75-81fd480f36a8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:20.570254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 38/81] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:20.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c698ad-851d-419c-82a3-44835df2b633 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:20.596081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:20.600235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:20.641055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0f73a-9604-48ee-9a75-81fd480f36a8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:20.680084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fa5f3-10fb-496f-9ce9-82fc7619f5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:20.680898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fa5f3-10fb-496f-9ce9-82fc7619f5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:20.681212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fa5f3-10fb-496f-9ce9-82fc7619f5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:20.681467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fa5f3-10fb-496f-9ce9-82fc7619f5d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:20.692706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983fa5f3-10fb-496f-9ce9-82fc7619f5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:20.693922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c698ad-851d-419c-82a3-44835df2b633 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-983fa5f3-10fb-496f-9ce9-82fc7619f5d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:20.701228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:20.702380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:20.703434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 44/82] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:20.704637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fa5f3-10fb-496f-9ce9-82fc7619f5d2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:20.743891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca8276-c9b3-428f-9d3d-1c88de6fba87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:20.745327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca8276-c9b3-428f-9d3d-1c88de6fba87 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:20.745835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca8276-c9b3-428f-9d3d-1c88de6fba87 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:20.746484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca8276-c9b3-428f-9d3d-1c88de6fba87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:20.842074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fa5f3-10fb-496f-9ce9-82fc7619f5d2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:20.853955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca8276-c9b3-428f-9d3d-1c88de6fba87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:20.855663 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca8276-c9b3-428f-9d3d-1c88de6fba87 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:20.875027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:20.885355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a8730-c7dc-46ae-b20b-9953bd3f3908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:20.886049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a8730-c7dc-46ae-b20b-9953bd3f3908 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:20.886349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a8730-c7dc-46ae-b20b-9953bd3f3908 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:20.886666 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a8730-c7dc-46ae-b20b-9953bd3f3908 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:20.887419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a8730-c7dc-46ae-b20b-9953bd3f3908 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:20.888958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a8730-c7dc-46ae-b20b-9953bd3f3908 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:20.895874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:20.928561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 45/83] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:10:20 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:20.928561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a8730-c7dc-46ae-b20b-9953bd3f3908 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:21.004348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 39/84] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:21.005409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca8276-c9b3-428f-9d3d-1c88de6fba87 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:21.113789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a8730-c7dc-46ae-b20b-9953bd3f3908 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:21.177447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ca693-b7f4-45af-8793-901840c1129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:21.178404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ca693-b7f4-45af-8793-901840c1129e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:21.178639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca8276-c9b3-428f-9d3d-1c88de6fba87 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:21.178933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ca693-b7f4-45af-8793-901840c1129e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:21.179226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ca693-b7f4-45af-8793-901840c1129e None admin] PATH_INFO: `/v3/users/c5ec92316a2c452dab52aa49d7256f2f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:21.180626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ca693-b7f4-45af-8793-901840c1129e None admin] RBAC: Authorizing `identity:get_user(user_id=c5ec92316a2c452dab52aa49d7256f2f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:21.183682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ca693-b7f4-45af-8793-901840c1129e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:21.185857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 46/85] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:20 2022] GET /identity/v3/users/c5ec92316a2c452dab52aa49d7256f2f => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:21.186465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ca693-b7f4-45af-8793-901840c1129e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:21.217982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:21.241597 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cf013-300e-491a-931f-98926e848567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:21.243237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cf013-300e-491a-931f-98926e848567 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:21.243768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cf013-300e-491a-931f-98926e848567 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:21.244081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38cf013-300e-491a-931f-98926e848567 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:21.248000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cf013-300e-491a-931f-98926e848567 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:21.251110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38cf013-300e-491a-931f-98926e848567 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:21.254480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 40/86] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:21 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:21.260474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cf013-300e-491a-931f-98926e848567 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:21.271073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:21.375980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ca693-b7f4-45af-8793-901840c1129e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:21.461079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe065b-a017-4767-995d-c45126ca23dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:21.461878 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe065b-a017-4767-995d-c45126ca23dd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:21.462405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe065b-a017-4767-995d-c45126ca23dd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:21.462966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfe065b-a017-4767-995d-c45126ca23dd None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:21.464304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe065b-a017-4767-995d-c45126ca23dd None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:21.464861 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe065b-a017-4767-995d-c45126ca23dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:21.467629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfe065b-a017-4767-995d-c45126ca23dd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:21.485110 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 47/87] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:21.486364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe065b-a017-4767-995d-c45126ca23dd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:21.494891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38cf013-300e-491a-931f-98926e848567 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:21.569139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca9bef-be14-4aeb-ac8a-f66db0ec6ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:21.572297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca9bef-be14-4aeb-ac8a-f66db0ec6ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:21.573298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca9bef-be14-4aeb-ac8a-f66db0ec6ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:21.574075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca9bef-be14-4aeb-ac8a-f66db0ec6ae6 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:21.577021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca9bef-be14-4aeb-ac8a-f66db0ec6ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:21.582280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca9bef-be14-4aeb-ac8a-f66db0ec6ae6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:21.587756 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 41/88] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:21 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:21.589920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca9bef-be14-4aeb-ac8a-f66db0ec6ae6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:21.696475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfe065b-a017-4767-995d-c45126ca23dd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:21.722670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:21.731988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:21.772152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd1eb6-e632-42ab-b185-dad6389a25f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:21.773359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd1eb6-e632-42ab-b185-dad6389a25f3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:21.773849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd1eb6-e632-42ab-b185-dad6389a25f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:21.774370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd1eb6-e632-42ab-b185-dad6389a25f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:21.819046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca9bef-be14-4aeb-ac8a-f66db0ec6ae6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:21.875864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd1eb6-e632-42ab-b185-dad6389a25f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:21.878331 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd1eb6-e632-42ab-b185-dad6389a25f3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:21.894895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1648a5-a4a9-459d-9a86-46144e77ed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:21.898506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1648a5-a4a9-459d-9a86-46144e77ed81 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:21.899443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1648a5-a4a9-459d-9a86-46144e77ed81 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:21.900233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1648a5-a4a9-459d-9a86-46144e77ed81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:21.902625 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1648a5-a4a9-459d-9a86-46144e77ed81 None admin] RBAC: Adding query filter params (role.id=ef48ab9c5b844be59351a260130d3071, scope.project.id=5902af097cc346559cbefdb217f7394c, user.id=48574d0c8c5d4648b0a8c6d7d6d34287) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:21.903508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1648a5-a4a9-459d-9a86-46144e77ed81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:21.906810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1648a5-a4a9-459d-9a86-46144e77ed81 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:21.920181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 42/89] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:21 2022] GET /identity/v3/role_assignments?user.id=48574d0c8c5d4648b0a8c6d7d6d34287&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=ef48ab9c5b844be59351a260130d3071 => generated 259 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:21.922438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1648a5-a4a9-459d-9a86-46144e77ed81 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:22.015581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 48/90] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:22.016225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd1eb6-e632-42ab-b185-dad6389a25f3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:22.038127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:22.152650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1648a5-a4a9-459d-9a86-46144e77ed81 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:22.180223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd1eb6-e632-42ab-b185-dad6389a25f3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:22.192415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:22.210806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4afd0-5062-4089-85e9-0ccf1990c6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:22.212339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4afd0-5062-4089-85e9-0ccf1990c6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:22.212914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4afd0-5062-4089-85e9-0ccf1990c6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:22.213635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e4afd0-5062-4089-85e9-0ccf1990c6d7 None admin] PATH_INFO: `/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:22.215496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4afd0-5062-4089-85e9-0ccf1990c6d7 None admin] RBAC: Authorizing `identity:get_user(user_id=48574d0c8c5d4648b0a8c6d7d6d34287)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:22.217379 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e4afd0-5062-4089-85e9-0ccf1990c6d7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:22.219858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 43/91] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:21 2022] GET /identity/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287 => generated 275 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:22.221857 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4afd0-5062-4089-85e9-0ccf1990c6d7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:22.242555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68772950-2019-423a-8798-74dfbb7d0986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:22.243338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68772950-2019-423a-8798-74dfbb7d0986 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:22.243632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68772950-2019-423a-8798-74dfbb7d0986 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:22.243940 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68772950-2019-423a-8798-74dfbb7d0986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:22.246853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68772950-2019-423a-8798-74dfbb7d0986 None admin] RBAC: Adding query filter params (role.id=1d508b4effb546039e87dd273933af16, scope.project.id=5902af097cc346559cbefdb217f7394c, user.id=48574d0c8c5d4648b0a8c6d7d6d34287) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:22.247270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68772950-2019-423a-8798-74dfbb7d0986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:22.249023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68772950-2019-423a-8798-74dfbb7d0986 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:22.258294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 49/92] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:22 2022] GET /identity/v3/role_assignments?user.id=48574d0c8c5d4648b0a8c6d7d6d34287&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=1d508b4effb546039e87dd273933af16 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:22.259709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68772950-2019-423a-8798-74dfbb7d0986 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:22.363949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e4afd0-5062-4089-85e9-0ccf1990c6d7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:22.365727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:22.395618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68772950-2019-423a-8798-74dfbb7d0986 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:22.410975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21eb700-02e7-4ea8-a855-9c658a761649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:22.412925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eb700-02e7-4ea8-a855-9c658a761649 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:22.413576 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eb700-02e7-4ea8-a855-9c658a761649 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:22.414108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21eb700-02e7-4ea8-a855-9c658a761649 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:22.415871 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21eb700-02e7-4ea8-a855-9c658a761649 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:22.417872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21eb700-02e7-4ea8-a855-9c658a761649 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:22.420305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 44/93] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:22 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:22.430096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21eb700-02e7-4ea8-a855-9c658a761649 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:22.445176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17754395-0ac9-44b8-baf5-0780061c3ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:22.446291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17754395-0ac9-44b8-baf5-0780061c3ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:22.446692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17754395-0ac9-44b8-baf5-0780061c3ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:22.447044 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17754395-0ac9-44b8-baf5-0780061c3ba7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:22.526069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:22.549136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17754395-0ac9-44b8-baf5-0780061c3ba7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:22.551156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17754395-0ac9-44b8-baf5-0780061c3ba7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:22.580809 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21eb700-02e7-4ea8-a855-9c658a761649 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:22.623415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7e22c-0a04-4b2d-9cb9-1013b78eaeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:22.624213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7e22c-0a04-4b2d-9cb9-1013b78eaeda None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:22.624610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7e22c-0a04-4b2d-9cb9-1013b78eaeda None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:22.624967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7e22c-0a04-4b2d-9cb9-1013b78eaeda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:22.625901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7e22c-0a04-4b2d-9cb9-1013b78eaeda None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=5902af097cc346559cbefdb217f7394c, user.id=c5ec92316a2c452dab52aa49d7256f2f) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:22.626294 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7e22c-0a04-4b2d-9cb9-1013b78eaeda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:22.627891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7e22c-0a04-4b2d-9cb9-1013b78eaeda None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:22.639233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 45/94] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:22 2022] GET /identity/v3/role_assignments?user.id=c5ec92316a2c452dab52aa49d7256f2f&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:22.675344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:22.689023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 50/95] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:22.699325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7e22c-0a04-4b2d-9cb9-1013b78eaeda None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:22.857903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7e22c-0a04-4b2d-9cb9-1013b78eaeda None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:22.909751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:22.920583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ddd1d-05a5-4dee-97a4-be96ff7fd456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:22.921489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ddd1d-05a5-4dee-97a4-be96ff7fd456 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:22.921781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ddd1d-05a5-4dee-97a4-be96ff7fd456 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:22.922068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ddd1d-05a5-4dee-97a4-be96ff7fd456 None admin] PATH_INFO: `/v3/projects/663aac864cb941cc8165cc02ae504cf6` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:22.923494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ddd1d-05a5-4dee-97a4-be96ff7fd456 None admin] RBAC: Authorizing `identity:get_project(project_id=663aac864cb941cc8165cc02ae504cf6)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:22.925110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ddd1d-05a5-4dee-97a4-be96ff7fd456 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:22.927342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 46/96] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:22 2022] GET /identity/v3/projects/663aac864cb941cc8165cc02ae504cf6 => generated 312 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:22.936830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17754395-0ac9-44b8-baf5-0780061c3ba7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:23.109500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17754395-0ac9-44b8-baf5-0780061c3ba7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:23.171625 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f001e78-4e7b-4142-958e-889d082d9d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:23.172246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f001e78-4e7b-4142-958e-889d082d9d62 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:23.172546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f001e78-4e7b-4142-958e-889d082d9d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:23.172828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f001e78-4e7b-4142-958e-889d082d9d62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:23.173833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f001e78-4e7b-4142-958e-889d082d9d62 None admin] RBAC: Adding query filter params (role.id=ef48ab9c5b844be59351a260130d3071, scope.project.id=663aac864cb941cc8165cc02ae504cf6, user.id=48574d0c8c5d4648b0a8c6d7d6d34287) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:23.174262 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f001e78-4e7b-4142-958e-889d082d9d62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:23.175956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f001e78-4e7b-4142-958e-889d082d9d62 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:23.184665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 51/97] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:22 2022] GET /identity/v3/role_assignments?user.id=48574d0c8c5d4648b0a8c6d7d6d34287&scope.project.id=663aac864cb941cc8165cc02ae504cf6&role.id=ef48ab9c5b844be59351a260130d3071 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:23.997811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fbad8-8386-4508-900b-9355c33bf3a0 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:23.998948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fbad8-8386-4508-900b-9355c33bf3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:23.999581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2fbad8-8386-4508-900b-9355c33bf3a0 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:24.001478 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 47/98] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:24.013617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c62a4-985f-4737-8bf2-0ebef020a595 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:24.014569 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c62a4-985f-4737-8bf2-0ebef020a595 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:24.015118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c62a4-985f-4737-8bf2-0ebef020a595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:24.034939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b49c62a4-985f-4737-8bf2-0ebef020a595 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:24.036238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b49c62a4-985f-4737-8bf2-0ebef020a595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:24.078952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:24.124132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 52/99] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:24.154310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fbad8-8386-4508-900b-9355c33bf3a0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:24.319900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2fbad8-8386-4508-900b-9355c33bf3a0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:24.374134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e092b98-ce8f-484f-844f-adb28e902414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:24.375337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e092b98-ce8f-484f-844f-adb28e902414 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:24.375636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e092b98-ce8f-484f-844f-adb28e902414 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:24.375927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e092b98-ce8f-484f-844f-adb28e902414 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:24.377255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e092b98-ce8f-484f-844f-adb28e902414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:24.378787 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e092b98-ce8f-484f-844f-adb28e902414 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:24.380646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 48/100] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:24.387372 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c62a4-985f-4737-8bf2-0ebef020a595 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:24.510342 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c62a4-985f-4737-8bf2-0ebef020a595 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:24.552668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928217a-94bd-4ed2-83ab-0642c91f1da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:24.553352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928217a-94bd-4ed2-83ab-0642c91f1da3 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:24.553808 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928217a-94bd-4ed2-83ab-0642c91f1da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:24.554100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928217a-94bd-4ed2-83ab-0642c91f1da3 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:24.554885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928217a-94bd-4ed2-83ab-0642c91f1da3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:24.556625 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928217a-94bd-4ed2-83ab-0642c91f1da3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:24.596469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 53/101] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:10:24 2022] POST /identity/v3/projects => generated 302 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:24.796305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db3364-8548-4ef7-b9ce-baa4dfe3ff82 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:24.796805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db3364-8548-4ef7-b9ce-baa4dfe3ff82 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:24.797314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db3364-8548-4ef7-b9ce-baa4dfe3ff82 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:24.800251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 49/102] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:24.809277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54f85c-6e70-4e08-88db-6d248229e0da None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:24.809566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54f85c-6e70-4e08-88db-6d248229e0da None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:24.809849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b54f85c-6e70-4e08-88db-6d248229e0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:24.839133 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3b54f85c-6e70-4e08-88db-6d248229e0da None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:24.841347 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3b54f85c-6e70-4e08-88db-6d248229e0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:24.880092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:24.985708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 54/103] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:25.031791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db3364-8548-4ef7-b9ce-baa4dfe3ff82 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:25.119828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:25.176414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db3364-8548-4ef7-b9ce-baa4dfe3ff82 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:25.229253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb88d8-63ec-4f95-8ac2-57f170532599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:25.230142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb88d8-63ec-4f95-8ac2-57f170532599 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:25.230432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb88d8-63ec-4f95-8ac2-57f170532599 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:25.230741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb88d8-63ec-4f95-8ac2-57f170532599 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:25.247847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb88d8-63ec-4f95-8ac2-57f170532599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-84fb88d8-63ec-4f95-8ac2-57f170532599 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:25.256855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:25.258504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:25.260021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:25.266097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 50/104] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:25.273726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54f85c-6e70-4e08-88db-6d248229e0da None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:25.545021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b54f85c-6e70-4e08-88db-6d248229e0da None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:25.606004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ae0fc-c346-436f-86cc-bd2a74e08f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:25.606733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ae0fc-c346-436f-86cc-bd2a74e08f21 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:25.607006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ae0fc-c346-436f-86cc-bd2a74e08f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:25.607296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ae0fc-c346-436f-86cc-bd2a74e08f21 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:25.608076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ae0fc-c346-436f-86cc-bd2a74e08f21 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:25.608397 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ae0fc-c346-436f-86cc-bd2a74e08f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:25.609850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ae0fc-c346-436f-86cc-bd2a74e08f21 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:25.627549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 55/105] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:25.636830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb88d8-63ec-4f95-8ac2-57f170532599 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:25.818820 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb88d8-63ec-4f95-8ac2-57f170532599 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:25.840205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096d11d-f821-4dee-94c7-f2fa1d939e6a None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:25.840620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096d11d-f821-4dee-94c7-f2fa1d939e6a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:25.840800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3096d11d-f821-4dee-94c7-f2fa1d939e6a None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:25.842295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 56/106] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:10:25.848651 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a21e70-0fae-486e-a298-474df50fd713 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:25.848882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:25.849019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a21e70-0fae-486e-a298-474df50fd713 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:25.850114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a21e70-0fae-486e-a298-474df50fd713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:25.880375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac0759-911c-4a92-b71b-afbf376c6d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:25.881917 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac0759-911c-4a92-b71b-afbf376c6d71 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:25.882209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac0759-911c-4a92-b71b-afbf376c6d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:25.882493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac0759-911c-4a92-b71b-afbf376c6d71 None admin] PATH_INFO: `/v3/users/ff084582e0db4ebe8c66948ef96a56da` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:25.898179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-69a21e70-0fae-486e-a298-474df50fd713 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:25.899483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-69a21e70-0fae-486e-a298-474df50fd713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:25.921283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:25.930987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac0759-911c-4a92-b71b-afbf376c6d71 None admin] RBAC: Authorizing `identity:get_user(user_id=ff084582e0db4ebe8c66948ef96a56da)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:25.933403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac0759-911c-4a92-b71b-afbf376c6d71 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:25.935346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 51/107] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:25 2022] GET /identity/v3/users/ff084582e0db4ebe8c66948ef96a56da => generated 289 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:25.942242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac0759-911c-4a92-b71b-afbf376c6d71 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:26.122125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 57/108] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:26.134871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895c665-672c-4925-beb8-ba941ec1072e None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:26.135289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895c665-672c-4925-beb8-ba941ec1072e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:26.135516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895c665-672c-4925-beb8-ba941ec1072e None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:26.136862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 58/109] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:10:26.142687 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62d464-46c3-44a2-9837-fd7b4c344913 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:26.143056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62d464-46c3-44a2-9837-fd7b4c344913 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:26.143297 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62d464-46c3-44a2-9837-fd7b4c344913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:26.161956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7a62d464-46c3-44a2-9837-fd7b4c344913 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:26.163433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7a62d464-46c3-44a2-9837-fd7b4c344913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:26.246748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 59/110] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:26.247894 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62d464-46c3-44a2-9837-fd7b4c344913 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:26.277817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac0759-911c-4a92-b71b-afbf376c6d71 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:26.346982 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:26.376388 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62d464-46c3-44a2-9837-fd7b4c344913 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:26.397538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186235e-2410-4e1f-9049-c27f7351f95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:26.398221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186235e-2410-4e1f-9049-c27f7351f95e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:26.398519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186235e-2410-4e1f-9049-c27f7351f95e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:26.398812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9186235e-2410-4e1f-9049-c27f7351f95e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:26.417022 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4f97f-7457-44e2-9449-1aca90972fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:26.417848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4f97f-7457-44e2-9449-1aca90972fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:26.418127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4f97f-7457-44e2-9449-1aca90972fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:26.418410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4f97f-7457-44e2-9449-1aca90972fd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:26.431081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4f97f-7457-44e2-9449-1aca90972fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-cde4f97f-7457-44e2-9449-1aca90972fd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:26.439466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:26.443610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 60/111] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:26.446766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4f97f-7457-44e2-9449-1aca90972fd3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:26.521107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:26.582449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186235e-2410-4e1f-9049-c27f7351f95e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:26.584136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9186235e-2410-4e1f-9049-c27f7351f95e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:26.593491 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4f97f-7457-44e2-9449-1aca90972fd3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:26.668085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711fd2f-a847-4768-8eb1-4bd75e9a9b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:26.668839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711fd2f-a847-4768-8eb1-4bd75e9a9b27 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:26.669125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711fd2f-a847-4768-8eb1-4bd75e9a9b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:26.669410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711fd2f-a847-4768-8eb1-4bd75e9a9b27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:26.679017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:26.688130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711fd2f-a847-4768-8eb1-4bd75e9a9b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a711fd2f-a847-4768-8eb1-4bd75e9a9b27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:26.698095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:26.699840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:26.701336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 61/112] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:10:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:26.715104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711fd2f-a847-4768-8eb1-4bd75e9a9b27 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:26.882224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 52/113] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 940 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:26.886761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186235e-2410-4e1f-9049-c27f7351f95e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:26.891501 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711fd2f-a847-4768-8eb1-4bd75e9a9b27 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:26.939025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b596aa5-46ef-4aa0-866b-dfe396d398ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:26.939788 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b596aa5-46ef-4aa0-866b-dfe396d398ac None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:26.940087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b596aa5-46ef-4aa0-866b-dfe396d398ac None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:26.940469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b596aa5-46ef-4aa0-866b-dfe396d398ac None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:26.941263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b596aa5-46ef-4aa0-866b-dfe396d398ac None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:26.941598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b596aa5-46ef-4aa0-866b-dfe396d398ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:26.943698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b596aa5-46ef-4aa0-866b-dfe396d398ac None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:26.955717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 62/114] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:26.956252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b596aa5-46ef-4aa0-866b-dfe396d398ac None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:27.065007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:27.116194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9186235e-2410-4e1f-9049-c27f7351f95e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:27.121135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:27.198204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b596aa5-46ef-4aa0-866b-dfe396d398ac None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:27.200197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e53cca-a02d-4041-b7c9-709e5584f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:27.201604 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e53cca-a02d-4041-b7c9-709e5584f9cb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:27.202261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e53cca-a02d-4041-b7c9-709e5584f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:27.202958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e53cca-a02d-4041-b7c9-709e5584f9cb None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:27.204631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e53cca-a02d-4041-b7c9-709e5584f9cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:27.205324 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e53cca-a02d-4041-b7c9-709e5584f9cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:27.208277 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e53cca-a02d-4041-b7c9-709e5584f9cb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:27.226160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 53/115] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:10:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:27.232135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e53cca-a02d-4041-b7c9-709e5584f9cb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:27.250945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dd46a-ae51-463f-9e23-7963f70df047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:27.251936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd46a-ae51-463f-9e23-7963f70df047 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:27.252229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd46a-ae51-463f-9e23-7963f70df047 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:27.252515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd46a-ae51-463f-9e23-7963f70df047 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:27.254033 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dd46a-ae51-463f-9e23-7963f70df047 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:27.255861 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dd46a-ae51-463f-9e23-7963f70df047 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:27.259149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 63/116] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:26 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:27.263155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89db4d-3ca1-4bdb-93c5-4451ee2ebb59 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:27.263758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89db4d-3ca1-4bdb-93c5-4451ee2ebb59 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:27.264263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89db4d-3ca1-4bdb-93c5-4451ee2ebb59 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:27.272400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 64/117] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:27 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:27.273143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89db4d-3ca1-4bdb-93c5-4451ee2ebb59 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:27.414205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e53cca-a02d-4041-b7c9-709e5584f9cb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:27.445361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae89db4d-3ca1-4bdb-93c5-4451ee2ebb59 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:27.461421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd08d8-c5a0-4818-a3b8-ab7840b50fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:27.462715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd08d8-c5a0-4818-a3b8-ab7840b50fab None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:27.463010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd08d8-c5a0-4818-a3b8-ab7840b50fab None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:27.463291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd08d8-c5a0-4818-a3b8-ab7840b50fab None admin] PATH_INFO: `/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:27.465007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd08d8-c5a0-4818-a3b8-ab7840b50fab None admin] RBAC: Authorizing `identity:get_user(user_id=48574d0c8c5d4648b0a8c6d7d6d34287)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:27.466706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd08d8-c5a0-4818-a3b8-ab7840b50fab None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:27.468443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 54/118] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:27 2022] GET /identity/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:27.472024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd08d8-c5a0-4818-a3b8-ab7840b50fab None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:27.484131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:27.486453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:27.507731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efafcee-3880-4653-bb79-bc15c0d94c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:27.508954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efafcee-3880-4653-bb79-bc15c0d94c1b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:27.509468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efafcee-3880-4653-bb79-bc15c0d94c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:27.509971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efafcee-3880-4653-bb79-bc15c0d94c1b None admin] PATH_INFO: `/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:27.512039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efafcee-3880-4653-bb79-bc15c0d94c1b None admin] RBAC: Authorizing `identity:get_user(user_id=48574d0c8c5d4648b0a8c6d7d6d34287)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:27.514806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efafcee-3880-4653-bb79-bc15c0d94c1b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:27.517866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 65/119] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:27 2022] GET /identity/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:27.520445 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7feab4-675c-42dd-b771-f10df2852b3a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:27.521627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7feab4-675c-42dd-b771-f10df2852b3a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:27.522159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7feab4-675c-42dd-b771-f10df2852b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:27.549962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3e7feab4-675c-42dd-b771-f10df2852b3a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:27.551416 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3e7feab4-675c-42dd-b771-f10df2852b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:27.608382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd08d8-c5a0-4818-a3b8-ab7840b50fab None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:27.640314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 66/120] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:27.645013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b4d50-ebae-424b-8663-6602ff1a4830 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:27.645013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b4d50-ebae-424b-8663-6602ff1a4830 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:27.645013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b4d50-ebae-424b-8663-6602ff1a4830 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:27.645013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 67/121] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:27.645013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b4d50-ebae-424b-8663-6602ff1a4830 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:27.674837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522f057-6dd7-4445-98bd-472cb4482c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:27.675636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522f057-6dd7-4445-98bd-472cb4482c54 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:27.675975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522f057-6dd7-4445-98bd-472cb4482c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:27.676373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522f057-6dd7-4445-98bd-472cb4482c54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:27.678092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522f057-6dd7-4445-98bd-472cb4482c54 None admin] RBAC: Adding query filter params (role.id=53b551bc12e84268991ca46bce5da0ec, scope.project.id=5902af097cc346559cbefdb217f7394c, user.id=ff084582e0db4ebe8c66948ef96a56da) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:27.678485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522f057-6dd7-4445-98bd-472cb4482c54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:27.680526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522f057-6dd7-4445-98bd-472cb4482c54 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:27.688026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 55/122] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:27 2022] GET /identity/v3/role_assignments?user.id=ff084582e0db4ebe8c66948ef96a56da&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=53b551bc12e84268991ca46bce5da0ec => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:27.688899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522f057-6dd7-4445-98bd-472cb4482c54 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:27.804897 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:27.814952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b4d50-ebae-424b-8663-6602ff1a4830 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:27.846324 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522f057-6dd7-4445-98bd-472cb4482c54 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:27.860572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:27.880728 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dbd79-6ec3-4436-a69f-c99d042b647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:27.881941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dbd79-6ec3-4436-a69f-c99d042b647d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:27.882325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dbd79-6ec3-4436-a69f-c99d042b647d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:27.882649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dbd79-6ec3-4436-a69f-c99d042b647d None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:27.884030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0dbd79-6ec3-4436-a69f-c99d042b647d None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:27.885548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0dbd79-6ec3-4436-a69f-c99d042b647d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:27.887406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 68/123] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:27 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:27.888649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbed12d-7b37-479b-b62a-aef87a740f30 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:27.888943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbed12d-7b37-479b-b62a-aef87a740f30 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:27.889221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbed12d-7b37-479b-b62a-aef87a740f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:27.903257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c395072-8550-4f33-a8b8-bdcbf5c21037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:27.904814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c395072-8550-4f33-a8b8-bdcbf5c21037 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:27.905310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c395072-8550-4f33-a8b8-bdcbf5c21037 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:27.905786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c395072-8550-4f33-a8b8-bdcbf5c21037 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:27.908304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c395072-8550-4f33-a8b8-bdcbf5c21037 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:27.908823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3dbed12d-7b37-479b-b62a-aef87a740f30 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:27.910286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c395072-8550-4f33-a8b8-bdcbf5c21037 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:27.910951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3dbed12d-7b37-479b-b62a-aef87a740f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:27.912598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 56/124] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:27 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:27.914149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c395072-8550-4f33-a8b8-bdcbf5c21037 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:28.026320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 69/125] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:28.027529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbed12d-7b37-479b-b62a-aef87a740f30 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:28.072907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c395072-8550-4f33-a8b8-bdcbf5c21037 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:28.122922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9024d8-9a66-4ce5-aaa2-21e06816fcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:28.124021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9024d8-9a66-4ce5-aaa2-21e06816fcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:28.124315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9024d8-9a66-4ce5-aaa2-21e06816fcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:28.124596 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9024d8-9a66-4ce5-aaa2-21e06816fcb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:28.138257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:28.146083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9024d8-9a66-4ce5-aaa2-21e06816fcb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-7f9024d8-9a66-4ce5-aaa2-21e06816fcb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:28.157920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:28.158919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 57/126] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:10:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:28.160189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9024d8-9a66-4ce5-aaa2-21e06816fcb9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:28.194624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbed12d-7b37-479b-b62a-aef87a740f30 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:28.237940 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:28.261712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b620e23-9fe3-49ce-9763-4e96945eac00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:28.263502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620e23-9fe3-49ce-9763-4e96945eac00 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:28.263944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620e23-9fe3-49ce-9763-4e96945eac00 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:28.264386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b620e23-9fe3-49ce-9763-4e96945eac00 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c/users/48574d0c8c5d4648b0a8c6d7d6d34287/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:28.267356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b620e23-9fe3-49ce-9763-4e96945eac00 None admin] RBAC: Authorizing `identity:create_grant(project_id=5902af097cc346559cbefdb217f7394c, user_id=48574d0c8c5d4648b0a8c6d7d6d34287, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:28.269783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b620e23-9fe3-49ce-9763-4e96945eac00 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:28.291238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 70/127] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:10:28 2022] PUT /identity/v3/projects/5902af097cc346559cbefdb217f7394c/users/48574d0c8c5d4648b0a8c6d7d6d34287/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:10:28.292393 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b620e23-9fe3-49ce-9763-4e96945eac00 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:28.315013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9024d8-9a66-4ce5-aaa2-21e06816fcb9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:28.400621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4e99e-335a-4311-ac35-29b7d50f3547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:28.403551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4e99e-335a-4311-ac35-29b7d50f3547 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:28.404417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4e99e-335a-4311-ac35-29b7d50f3547 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:28.405476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f4e99e-335a-4311-ac35-29b7d50f3547 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c/users/48574d0c8c5d4648b0a8c6d7d6d34287/roles/1d508b4effb546039e87dd273933af16` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:28.411451 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4e99e-335a-4311-ac35-29b7d50f3547 None admin] RBAC: Authorizing `identity:create_grant(project_id=5902af097cc346559cbefdb217f7394c, user_id=48574d0c8c5d4648b0a8c6d7d6d34287, role_id=1d508b4effb546039e87dd273933af16)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:28.414679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f4e99e-335a-4311-ac35-29b7d50f3547 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:28.451079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 58/128] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:10:28 2022] PUT /identity/v3/projects/5902af097cc346559cbefdb217f7394c/users/48574d0c8c5d4648b0a8c6d7d6d34287/roles/1d508b4effb546039e87dd273933af16 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:10:28.453880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4e99e-335a-4311-ac35-29b7d50f3547 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:28.490463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b620e23-9fe3-49ce-9763-4e96945eac00 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:28.565248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15973a71-c4a0-42a4-ab1d-d01d6c28782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:28.566703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15973a71-c4a0-42a4-ab1d-d01d6c28782f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:28.567197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15973a71-c4a0-42a4-ab1d-d01d6c28782f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:28.567742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15973a71-c4a0-42a4-ab1d-d01d6c28782f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:28.591875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:28.592160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15973a71-c4a0-42a4-ab1d-d01d6c28782f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-15973a71-c4a0-42a4-ab1d-d01d6c28782f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:28.608599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:28.610286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 71/129] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:28.619722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03087-decd-4a4c-ab16-cd2af2964a1d None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:28.621053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03087-decd-4a4c-ab16-cd2af2964a1d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:28.621053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd03087-decd-4a4c-ab16-cd2af2964a1d None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:28.621053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 72/130] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:28.626828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b7ab2-b7b9-41d7-bb1b-a8067d166919 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:28.627248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b7ab2-b7b9-41d7-bb1b-a8067d166919 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:28.627665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b7ab2-b7b9-41d7-bb1b-a8067d166919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:28.656575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ec3b7ab2-b7b9-41d7-bb1b-a8067d166919 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:28.657714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ec3b7ab2-b7b9-41d7-bb1b-a8067d166919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:28.723388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f4e99e-335a-4311-ac35-29b7d50f3547 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:28.732453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:28.772952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 73/131] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:28.774201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b7ab2-b7b9-41d7-bb1b-a8067d166919 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:28.798928 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cd91b-e922-4eba-90fd-c59ca6fbc9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:28.799638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cd91b-e922-4eba-90fd-c59ca6fbc9bd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:28.799989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cd91b-e922-4eba-90fd-c59ca6fbc9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:28.800425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cd91b-e922-4eba-90fd-c59ca6fbc9bd None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:28.801296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cd91b-e922-4eba-90fd-c59ca6fbc9bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:28.801626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cd91b-e922-4eba-90fd-c59ca6fbc9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:28.803887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cd91b-e922-4eba-90fd-c59ca6fbc9bd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:28.829046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 59/132] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:28.835288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cd91b-e922-4eba-90fd-c59ca6fbc9bd None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:28.981628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b7ab2-b7b9-41d7-bb1b-a8067d166919 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:29.047024 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19310d3b-ba98-4f06-b4e7-1b57f77c09c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:29.048272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19310d3b-ba98-4f06-b4e7-1b57f77c09c4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:29.048789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19310d3b-ba98-4f06-b4e7-1b57f77c09c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:29.049278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19310d3b-ba98-4f06-b4e7-1b57f77c09c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:29.050332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19310d3b-ba98-4f06-b4e7-1b57f77c09c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:29.054280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19310d3b-ba98-4f06-b4e7-1b57f77c09c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:29.056281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19310d3b-ba98-4f06-b4e7-1b57f77c09c4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:29.072134 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 74/133] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:29.073613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19310d3b-ba98-4f06-b4e7-1b57f77c09c4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:29.082627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cd91b-e922-4eba-90fd-c59ca6fbc9bd None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:29.119798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:29.141914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5057c-147b-4209-8d30-6349b9f8c83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:29.143470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5057c-147b-4209-8d30-6349b9f8c83c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:29.144234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5057c-147b-4209-8d30-6349b9f8c83c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:29.144726 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5057c-147b-4209-8d30-6349b9f8c83c None admin] PATH_INFO: `/v3/users/c5ec92316a2c452dab52aa49d7256f2f` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:29.147713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5057c-147b-4209-8d30-6349b9f8c83c None admin] RBAC: Authorizing `identity:get_user(user_id=c5ec92316a2c452dab52aa49d7256f2f)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:29.151141 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5057c-147b-4209-8d30-6349b9f8c83c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:29.152335 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:29.154842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 60/134] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:28 2022] GET /identity/v3/users/c5ec92316a2c452dab52aa49d7256f2f => generated 247 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:29.156481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5057c-147b-4209-8d30-6349b9f8c83c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:29.288136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19310d3b-ba98-4f06-b4e7-1b57f77c09c4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:29.334474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5057c-147b-4209-8d30-6349b9f8c83c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:29.350793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea724d45-28d2-4eab-be41-5d5b478199b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:29.351877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea724d45-28d2-4eab-be41-5d5b478199b2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:29.352400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea724d45-28d2-4eab-be41-5d5b478199b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:29.352732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea724d45-28d2-4eab-be41-5d5b478199b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:29.354730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea724d45-28d2-4eab-be41-5d5b478199b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:29.357738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea724d45-28d2-4eab-be41-5d5b478199b2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:29.361152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 75/135] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:29.362567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea724d45-28d2-4eab-be41-5d5b478199b2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:29.398553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f79bd-e6b8-4a8a-b3ab-eeae1708a81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:29.402207 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f79bd-e6b8-4a8a-b3ab-eeae1708a81b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:29.402513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f79bd-e6b8-4a8a-b3ab-eeae1708a81b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:29.402820 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f79bd-e6b8-4a8a-b3ab-eeae1708a81b None admin] PATH_INFO: `/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:29.407183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f79bd-e6b8-4a8a-b3ab-eeae1708a81b None admin] RBAC: Authorizing `identity:get_user(user_id=48574d0c8c5d4648b0a8c6d7d6d34287)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:29.415370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f79bd-e6b8-4a8a-b3ab-eeae1708a81b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:29.417482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 61/136] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:29 2022] GET /identity/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287 => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:29.419867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f79bd-e6b8-4a8a-b3ab-eeae1708a81b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:29.613173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:29.675682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:29.683335 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea724d45-28d2-4eab-be41-5d5b478199b2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:29.771805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f79bd-e6b8-4a8a-b3ab-eeae1708a81b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:29.795425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b46faf-c7de-4035-9847-54b3805d1d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:29.796628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b46faf-c7de-4035-9847-54b3805d1d87 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:29.796913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b46faf-c7de-4035-9847-54b3805d1d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:29.797193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b46faf-c7de-4035-9847-54b3805d1d87 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:29.803851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b46faf-c7de-4035-9847-54b3805d1d87 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:29.805493 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b46faf-c7de-4035-9847-54b3805d1d87 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:29.808067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 76/137] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:29 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:29.808722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b46faf-c7de-4035-9847-54b3805d1d87 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:29.833968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cb1b9-7df7-4981-90d0-4c46db8b7c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:29.835075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cb1b9-7df7-4981-90d0-4c46db8b7c6d None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:29.835593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cb1b9-7df7-4981-90d0-4c46db8b7c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:29.836126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cb1b9-7df7-4981-90d0-4c46db8b7c6d None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:29.837172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cb1b9-7df7-4981-90d0-4c46db8b7c6d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:29.839095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cb1b9-7df7-4981-90d0-4c46db8b7c6d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:29.869841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 62/138] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:10:29 2022] POST /identity/v3/users => generated 283 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:29.871094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cb1b9-7df7-4981-90d0-4c46db8b7c6d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:30.017757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cb1b9-7df7-4981-90d0-4c46db8b7c6d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:30.040510 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:30.073081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b348c3-efb4-480b-bebb-537d63355a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:30.074015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b348c3-efb4-480b-bebb-537d63355a50 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:30.074305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b348c3-efb4-480b-bebb-537d63355a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:30.074609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b348c3-efb4-480b-bebb-537d63355a50 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c/users/c5ec92316a2c452dab52aa49d7256f2f/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:30.076915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b348c3-efb4-480b-bebb-537d63355a50 None admin] RBAC: Authorizing `identity:create_grant(project_id=5902af097cc346559cbefdb217f7394c, user_id=c5ec92316a2c452dab52aa49d7256f2f, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:30.078419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b348c3-efb4-480b-bebb-537d63355a50 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:30.094549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 63/139] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:10:29 2022] PUT /identity/v3/projects/5902af097cc346559cbefdb217f7394c/users/c5ec92316a2c452dab52aa49d7256f2f/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:10:30.109077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b46faf-c7de-4035-9847-54b3805d1d87 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:30.179576 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b3d88-33a3-4da5-bf7b-c293c270d594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:30.180392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b3d88-33a3-4da5-bf7b-c293c270d594 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:30.180678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b3d88-33a3-4da5-bf7b-c293c270d594 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:30.180958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b3d88-33a3-4da5-bf7b-c293c270d594 None admin] PATH_INFO: `/v3/projects/663aac864cb941cc8165cc02ae504cf6` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:30.182307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b3d88-33a3-4da5-bf7b-c293c270d594 None admin] RBAC: Authorizing `identity:get_project(project_id=663aac864cb941cc8165cc02ae504cf6)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:30.183933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b3d88-33a3-4da5-bf7b-c293c270d594 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:30.185859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 77/140] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:29 2022] GET /identity/v3/projects/663aac864cb941cc8165cc02ae504cf6 => generated 312 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:30.192185 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b348c3-efb4-480b-bebb-537d63355a50 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:30.250463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4b068-2dd3-4343-9acd-67900d6dedcc None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:30.250976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4b068-2dd3-4343-9acd-67900d6dedcc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:30.251382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4b068-2dd3-4343-9acd-67900d6dedcc None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:30.253323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 78/141] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:30.262867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fb02f-7394-4487-a732-5097c315c312 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:30.263274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fb02f-7394-4487-a732-5097c315c312 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:30.263559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fb02f-7394-4487-a732-5097c315c312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:30.303803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-220fb02f-7394-4487-a732-5097c315c312 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:30.305101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-220fb02f-7394-4487-a732-5097c315c312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:30.380639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b348c3-efb4-480b-bebb-537d63355a50 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:30.407405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:30.484822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccfb1a-003b-4767-a1cc-e3ced85e7baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:30.487014 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccfb1a-003b-4767-a1cc-e3ced85e7baf None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:30.488833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccfb1a-003b-4767-a1cc-e3ced85e7baf None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:30.489147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccfb1a-003b-4767-a1cc-e3ced85e7baf None admin] PATH_INFO: `/v3/projects/663aac864cb941cc8165cc02ae504cf6/users/48574d0c8c5d4648b0a8c6d7d6d34287/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:30.493394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccfb1a-003b-4767-a1cc-e3ced85e7baf None admin] RBAC: Authorizing `identity:create_grant(project_id=663aac864cb941cc8165cc02ae504cf6, user_id=48574d0c8c5d4648b0a8c6d7d6d34287, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:30.494955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccfb1a-003b-4767-a1cc-e3ced85e7baf None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:30.508268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 79/142] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:30.518898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 64/143] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:10:30 2022] PUT /identity/v3/projects/663aac864cb941cc8165cc02ae504cf6/users/48574d0c8c5d4648b0a8c6d7d6d34287/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:10:30.542137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220fb02f-7394-4487-a732-5097c315c312 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:30.757994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220fb02f-7394-4487-a732-5097c315c312 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:30.836621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae372eae-618b-436e-965f-a97d271ade99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:30.837937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372eae-618b-436e-965f-a97d271ade99 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:30.838419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372eae-618b-436e-965f-a97d271ade99 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:30.838926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae372eae-618b-436e-965f-a97d271ade99 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:30.869337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae372eae-618b-436e-965f-a97d271ade99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ae372eae-618b-436e-965f-a97d271ade99 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:30.883999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:30.885646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:30.887120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 80/144] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:30.895536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccfb1a-003b-4767-a1cc-e3ced85e7baf None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:30.971877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:31.147671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccfb1a-003b-4767-a1cc-e3ced85e7baf None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:31.234724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9ce0f-90b1-483f-92ff-af93cebe189c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:31.236175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9ce0f-90b1-483f-92ff-af93cebe189c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:31.236683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9ce0f-90b1-483f-92ff-af93cebe189c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:31.237171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9ce0f-90b1-483f-92ff-af93cebe189c None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:31.238572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9ce0f-90b1-483f-92ff-af93cebe189c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:31.239104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9ce0f-90b1-483f-92ff-af93cebe189c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:31.242490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d9ce0f-90b1-483f-92ff-af93cebe189c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:31.269192 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 65/145] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:31.279103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae372eae-618b-436e-965f-a97d271ade99 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:31.347516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:31.506876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae372eae-618b-436e-965f-a97d271ade99 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:31.580617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05240ec-cd39-4854-a8cc-bc229d738a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:31.582248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05240ec-cd39-4854-a8cc-bc229d738a36 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:31.582817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05240ec-cd39-4854-a8cc-bc229d738a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:31.584047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05240ec-cd39-4854-a8cc-bc229d738a36 None admin] PATH_INFO: `/v3/users/ff084582e0db4ebe8c66948ef96a56da` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:31.587941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05240ec-cd39-4854-a8cc-bc229d738a36 None admin] RBAC: Authorizing `identity:get_user(user_id=ff084582e0db4ebe8c66948ef96a56da)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:31.591004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05240ec-cd39-4854-a8cc-bc229d738a36 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:31.594399 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 81/146] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:31 2022] GET /identity/v3/users/ff084582e0db4ebe8c66948ef96a56da => generated 289 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:31.604123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9ce0f-90b1-483f-92ff-af93cebe189c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:31.804995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f2b6c-f2df-4911-b000-0f1f7b23c342 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:31.806555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f2b6c-f2df-4911-b000-0f1f7b23c342 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:31.807395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f2b6c-f2df-4911-b000-0f1f7b23c342 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:31.811302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 82/147] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:31.821055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45843bb-f6be-4520-b3b4-55b09205d36c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:31.821981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45843bb-f6be-4520-b3b4-55b09205d36c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:31.822809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45843bb-f6be-4520-b3b4-55b09205d36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:31.852393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:31.870944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9ce0f-90b1-483f-92ff-af93cebe189c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:31.875767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d45843bb-f6be-4520-b3b4-55b09205d36c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:31.876997 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d45843bb-f6be-4520-b3b4-55b09205d36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:31.985386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d050a61-2c17-464d-ab68-6db43e8fda39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:31.987135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d050a61-2c17-464d-ab68-6db43e8fda39 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:31.987619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d050a61-2c17-464d-ab68-6db43e8fda39 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:31.988075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d050a61-2c17-464d-ab68-6db43e8fda39 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:31.991782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d050a61-2c17-464d-ab68-6db43e8fda39 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:31.994750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d050a61-2c17-464d-ab68-6db43e8fda39 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:31.999057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 66/148] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:31 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:32.001430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5dbc7-a589-4441-895f-42a98f76d5d2 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:32.001971 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5dbc7-a589-4441-895f-42a98f76d5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:32.002497 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5dbc7-a589-4441-895f-42a98f76d5d2 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:32.032337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 67/149] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:31 2022] GET /identity => generated 270 bytes in 32 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:32.036586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5dbc7-a589-4441-895f-42a98f76d5d2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:32.104271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 83/150] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:32.106541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4ea83-25fc-4afa-86bb-329b860240cb None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:32.107630 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4ea83-25fc-4afa-86bb-329b860240cb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:32.107788 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4ea83-25fc-4afa-86bb-329b860240cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:32.160461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f9d4ea83-25fc-4afa-86bb-329b860240cb None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:32.163278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f9d4ea83-25fc-4afa-86bb-329b860240cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:32.252567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:32.317086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5dbc7-a589-4441-895f-42a98f76d5d2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:32.409842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630f45d-7085-468d-a51c-2fdbc2d0bee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:32.410710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630f45d-7085-468d-a51c-2fdbc2d0bee7 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:32.411002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630f45d-7085-468d-a51c-2fdbc2d0bee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:32.411906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630f45d-7085-468d-a51c-2fdbc2d0bee7 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c/users/ff084582e0db4ebe8c66948ef96a56da/roles/53b551bc12e84268991ca46bce5da0ec` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:32.418959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630f45d-7085-468d-a51c-2fdbc2d0bee7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5902af097cc346559cbefdb217f7394c, user_id=ff084582e0db4ebe8c66948ef96a56da, role_id=53b551bc12e84268991ca46bce5da0ec)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:32.420447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630f45d-7085-468d-a51c-2fdbc2d0bee7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:32.441322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 84/151] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:32.442712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4ea83-25fc-4afa-86bb-329b860240cb None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:32.445678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 68/152] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:10:32 2022] PUT /identity/v3/projects/5902af097cc346559cbefdb217f7394c/users/ff084582e0db4ebe8c66948ef96a56da/roles/53b551bc12e84268991ca46bce5da0ec => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:10:32.510137 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630f45d-7085-468d-a51c-2fdbc2d0bee7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:32.556609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:32.703903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630f45d-7085-468d-a51c-2fdbc2d0bee7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:32.769533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222223b2-ccaa-4031-96eb-30fde7888a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:32.770373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222223b2-ccaa-4031-96eb-30fde7888a2f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:32.771526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222223b2-ccaa-4031-96eb-30fde7888a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:32.771907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222223b2-ccaa-4031-96eb-30fde7888a2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:32.795007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222223b2-ccaa-4031-96eb-30fde7888a2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-222223b2-ccaa-4031-96eb-30fde7888a2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:32.807024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:32.808791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:32.810367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 69/153] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:10:32.814879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222223b2-ccaa-4031-96eb-30fde7888a2f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:32.835813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4ea83-25fc-4afa-86bb-329b860240cb None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:32.951986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:32.957430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f143d-d498-4562-9c5f-f59abc14a105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:32.959472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f143d-d498-4562-9c5f-f59abc14a105 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:32.960014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f143d-d498-4562-9c5f-f59abc14a105 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:32.960464 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f143d-d498-4562-9c5f-f59abc14a105 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:32.980546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222223b2-ccaa-4031-96eb-30fde7888a2f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:32.991667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f143d-d498-4562-9c5f-f59abc14a105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a48f143d-d498-4562-9c5f-f59abc14a105 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:33.034492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:33.036391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:33.038217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 85/154] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:10:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 596 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:33.046827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f143d-d498-4562-9c5f-f59abc14a105 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:33.051309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7163f-c243-49ac-8142-42cff983312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:33.052799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7163f-c243-49ac-8142-42cff983312e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:33.053079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7163f-c243-49ac-8142-42cff983312e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:33.053381 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7163f-c243-49ac-8142-42cff983312e None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:33.054247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7163f-c243-49ac-8142-42cff983312e None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:33.054557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7163f-c243-49ac-8142-42cff983312e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:33.056044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7163f-c243-49ac-8142-42cff983312e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:33.072528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 70/155] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:33.078326 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7163f-c243-49ac-8142-42cff983312e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:33.102117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:33.234528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7163f-c243-49ac-8142-42cff983312e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:33.290811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74d595-b3fe-4e9a-8868-ef787013dcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:33.291762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74d595-b3fe-4e9a-8868-ef787013dcb3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:33.292043 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74d595-b3fe-4e9a-8868-ef787013dcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:33.292347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a74d595-b3fe-4e9a-8868-ef787013dcb3 None admin] PATH_INFO: `/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:33.295411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74d595-b3fe-4e9a-8868-ef787013dcb3 None admin] RBAC: Authorizing `identity:get_user(user_id=48574d0c8c5d4648b0a8c6d7d6d34287)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:33.297047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a74d595-b3fe-4e9a-8868-ef787013dcb3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:33.299081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 71/156] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:33 2022] GET /identity/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:33.306206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74d595-b3fe-4e9a-8868-ef787013dcb3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:33.329999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:33.414806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f143d-d498-4562-9c5f-f59abc14a105 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:33.546933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a74d595-b3fe-4e9a-8868-ef787013dcb3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:33.552109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c75fe9-714f-4f52-8219-12e58c5a57ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:33.553040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c75fe9-714f-4f52-8219-12e58c5a57ba None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:33.553540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c75fe9-714f-4f52-8219-12e58c5a57ba None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:33.554041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c75fe9-714f-4f52-8219-12e58c5a57ba None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:33.555573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c75fe9-714f-4f52-8219-12e58c5a57ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:33.556165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c75fe9-714f-4f52-8219-12e58c5a57ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:33.559009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c75fe9-714f-4f52-8219-12e58c5a57ba None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:33.582823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 86/157] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:10:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 540 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:33.594428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c75fe9-714f-4f52-8219-12e58c5a57ba None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:33.644135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e44ac0-8744-445b-b2fd-868c68fccf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:33.644930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e44ac0-8744-445b-b2fd-868c68fccf9d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:33.645235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e44ac0-8744-445b-b2fd-868c68fccf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:33.646349 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e44ac0-8744-445b-b2fd-868c68fccf9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:33.844461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e44ac0-8744-445b-b2fd-868c68fccf9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:33.846006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e44ac0-8744-445b-b2fd-868c68fccf9d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:33.874719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:33.903150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:34.026731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c75fe9-714f-4f52-8219-12e58c5a57ba None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:34.124125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 72/158] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 819 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 08:10:34.130335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e44ac0-8744-445b-b2fd-868c68fccf9d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:34.165344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013c7b1-59e2-48e0-b415-33dbf5f1fa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:34.167410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013c7b1-59e2-48e0-b415-33dbf5f1fa16 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:34.168259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013c7b1-59e2-48e0-b415-33dbf5f1fa16 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:34.169014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013c7b1-59e2-48e0-b415-33dbf5f1fa16 None admin] PATH_INFO: `/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:34.175846 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013c7b1-59e2-48e0-b415-33dbf5f1fa16 None admin] RBAC: Authorizing `identity:get_user(user_id=48574d0c8c5d4648b0a8c6d7d6d34287)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:34.179263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013c7b1-59e2-48e0-b415-33dbf5f1fa16 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:34.186888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 87/159] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:33 2022] GET /identity/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287 => generated 275 bytes in 594 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:34.212806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013c7b1-59e2-48e0-b415-33dbf5f1fa16 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:34.305216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e44ac0-8744-445b-b2fd-868c68fccf9d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:34.343532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:34.372856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf461443-3eb3-4779-836a-4cfa1234d0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:34.373797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461443-3eb3-4779-836a-4cfa1234d0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:34.374083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461443-3eb3-4779-836a-4cfa1234d0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:34.374437 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf461443-3eb3-4779-836a-4cfa1234d0d4 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:34.379478 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf461443-3eb3-4779-836a-4cfa1234d0d4 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:34.381822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf461443-3eb3-4779-836a-4cfa1234d0d4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:34.383744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 73/160] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:34 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:34.390194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf461443-3eb3-4779-836a-4cfa1234d0d4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:34.475313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013c7b1-59e2-48e0-b415-33dbf5f1fa16 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:34.555832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7d2a6-20d1-464d-abe0-430bfef2869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:34.556932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7d2a6-20d1-464d-abe0-430bfef2869f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:34.557428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7d2a6-20d1-464d-abe0-430bfef2869f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:34.557918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7d2a6-20d1-464d-abe0-430bfef2869f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:34.615437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:34.653894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf461443-3eb3-4779-836a-4cfa1234d0d4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:34.745599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7d2a6-20d1-464d-abe0-430bfef2869f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:34.748280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7d2a6-20d1-464d-abe0-430bfef2869f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:34.752350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af587fa-d845-4292-ace4-9fa26e12e14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:34.754128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af587fa-d845-4292-ace4-9fa26e12e14d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:34.755670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af587fa-d845-4292-ace4-9fa26e12e14d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:34.756422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af587fa-d845-4292-ace4-9fa26e12e14d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:34.758212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af587fa-d845-4292-ace4-9fa26e12e14d None admin] RBAC: Adding query filter params (role.id=ef48ab9c5b844be59351a260130d3071, scope.project.id=5902af097cc346559cbefdb217f7394c, user.id=48574d0c8c5d4648b0a8c6d7d6d34287) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:34.759031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af587fa-d845-4292-ace4-9fa26e12e14d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:34.761919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af587fa-d845-4292-ace4-9fa26e12e14d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:34.780898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 74/161] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:34 2022] GET /identity/v3/role_assignments?user.id=48574d0c8c5d4648b0a8c6d7d6d34287&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=ef48ab9c5b844be59351a260130d3071 => generated 608 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:34.950765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 88/162] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:34.957767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af587fa-d845-4292-ace4-9fa26e12e14d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:35.119209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:35.245140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af587fa-d845-4292-ace4-9fa26e12e14d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:35.330204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63eca2f-075d-40a4-a6dc-9efae9e5c589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:35.332274 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eca2f-075d-40a4-a6dc-9efae9e5c589 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:35.333111 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eca2f-075d-40a4-a6dc-9efae9e5c589 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:35.333908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eca2f-075d-40a4-a6dc-9efae9e5c589 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:35.336732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63eca2f-075d-40a4-a6dc-9efae9e5c589 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:35.340814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63eca2f-075d-40a4-a6dc-9efae9e5c589 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:35.346439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 75/163] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:34 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:35.354009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7d2a6-20d1-464d-abe0-430bfef2869f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:35.471451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:35.610837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7d2a6-20d1-464d-abe0-430bfef2869f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:35.703964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8257851-ebd4-4552-a0a8-88e123957ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:35.705072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257851-ebd4-4552-a0a8-88e123957ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:35.705570 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257851-ebd4-4552-a0a8-88e123957ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:35.706061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8257851-ebd4-4552-a0a8-88e123957ae4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:35.707795 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8257851-ebd4-4552-a0a8-88e123957ae4 None admin] RBAC: Adding query filter params (role.id=1d508b4effb546039e87dd273933af16, scope.project.id=5902af097cc346559cbefdb217f7394c, user.id=48574d0c8c5d4648b0a8c6d7d6d34287) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:35.708879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8257851-ebd4-4552-a0a8-88e123957ae4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:35.711820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8257851-ebd4-4552-a0a8-88e123957ae4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:35.725035 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 89/164] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:35 2022] GET /identity/v3/role_assignments?user.id=48574d0c8c5d4648b0a8c6d7d6d34287&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=1d508b4effb546039e87dd273933af16 => generated 608 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:36.200721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5703a8-2f74-414f-821e-52eccbc88fe1 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:36.202287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5703a8-2f74-414f-821e-52eccbc88fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:36.203100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5703a8-2f74-414f-821e-52eccbc88fe1 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:36.206079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 76/165] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:36.215763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5555c-999f-433d-9640-f7ed4db315c0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:36.217129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5555c-999f-433d-9640-f7ed4db315c0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:36.218280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5555c-999f-433d-9640-f7ed4db315c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:36.259966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6dd5555c-999f-433d-9640-f7ed4db315c0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:36.262612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6dd5555c-999f-433d-9640-f7ed4db315c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:36.434998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 90/166] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:36.503380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5703a8-2f74-414f-821e-52eccbc88fe1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:36.691781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5703a8-2f74-414f-821e-52eccbc88fe1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:36.730767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:36.746912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5deb3-6e12-4435-be83-c6abd2f3d5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:36.747736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5deb3-6e12-4435-be83-c6abd2f3d5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:36.748023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5deb3-6e12-4435-be83-c6abd2f3d5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:36.748396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5deb3-6e12-4435-be83-c6abd2f3d5b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:36.764950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5deb3-6e12-4435-be83-c6abd2f3d5b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-40f5deb3-6e12-4435-be83-c6abd2f3d5b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:36.773976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:36.775694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:36.778079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:36.780131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 77/167] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:10:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:36.789790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5555c-999f-433d-9640-f7ed4db315c0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:36.903783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:37.079122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5555c-999f-433d-9640-f7ed4db315c0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:37.169291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623fc62-2334-451a-aba9-48038da4b6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:37.170504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623fc62-2334-451a-aba9-48038da4b6fc None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.171023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623fc62-2334-451a-aba9-48038da4b6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.171509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623fc62-2334-451a-aba9-48038da4b6fc None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.172933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623fc62-2334-451a-aba9-48038da4b6fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:37.173480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623fc62-2334-451a-aba9-48038da4b6fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:37.176179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623fc62-2334-451a-aba9-48038da4b6fc None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:37.193677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f0314-92fa-40e2-93cf-2d54294eb1d1 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.194325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f0314-92fa-40e2-93cf-2d54294eb1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.194839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f0314-92fa-40e2-93cf-2d54294eb1d1 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.196524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 78/168] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:10:37.198880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 91/169] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:37.206980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f0314-92fa-40e2-93cf-2d54294eb1d1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:37.215061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54b600-7098-437f-986f-37a1bd12b308 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.215545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54b600-7098-437f-986f-37a1bd12b308 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.216011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e54b600-7098-437f-986f-37a1bd12b308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.245596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3e54b600-7098-437f-986f-37a1bd12b308 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:37.247613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3e54b600-7098-437f-986f-37a1bd12b308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:37.401339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 92/170] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:37.410874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be66e6-f7a8-482d-9ebb-fbfc304501cf None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.412134 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be66e6-f7a8-482d-9ebb-fbfc304501cf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.412626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be66e6-f7a8-482d-9ebb-fbfc304501cf None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.415790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 93/171] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:37 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:37.435392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376140e8-5da4-436c-b8ca-44c6b6eb520c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.435934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376140e8-5da4-436c-b8ca-44c6b6eb520c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.436595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376140e8-5da4-436c-b8ca-44c6b6eb520c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.478837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-376140e8-5da4-436c-b8ca-44c6b6eb520c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:37.481068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-376140e8-5da4-436c-b8ca-44c6b6eb520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:37.540243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:37.569958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f0314-92fa-40e2-93cf-2d54294eb1d1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:37.616185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 94/172] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:37.617461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376140e8-5da4-436c-b8ca-44c6b6eb520c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:37.636770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ad6aa-e4aa-4880-bf44-ea84426290cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:37.637792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ad6aa-e4aa-4880-bf44-ea84426290cf None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.638081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ad6aa-e4aa-4880-bf44-ea84426290cf None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.638368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ad6aa-e4aa-4880-bf44-ea84426290cf None admin] PATH_INFO: `/v3/users/c5ec92316a2c452dab52aa49d7256f2f` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.639774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ad6aa-e4aa-4880-bf44-ea84426290cf None admin] RBAC: Authorizing `identity:get_user(user_id=c5ec92316a2c452dab52aa49d7256f2f)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:37.641248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ad6aa-e4aa-4880-bf44-ea84426290cf None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:37.643216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 79/173] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:37 2022] GET /identity/v3/users/c5ec92316a2c452dab52aa49d7256f2f => generated 247 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:37.646745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd06153-5e80-493e-86dc-5fa8df69f842 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.646745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd06153-5e80-493e-86dc-5fa8df69f842 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.646745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd06153-5e80-493e-86dc-5fa8df69f842 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.646745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 80/174] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:37.648799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac087a50-7166-4e32-965f-72568e8e0997 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.648799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac087a50-7166-4e32-965f-72568e8e0997 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.648799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac087a50-7166-4e32-965f-72568e8e0997 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.649517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 81/175] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:37.654771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6aa7f-3029-46eb-9f3b-a3d015328bdc None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.654771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6aa7f-3029-46eb-9f3b-a3d015328bdc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.654771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6aa7f-3029-46eb-9f3b-a3d015328bdc None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.654771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 82/176] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:37.658762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b032c58-5375-4b12-a083-06f04551a68c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.659057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b032c58-5375-4b12-a083-06f04551a68c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.659390 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b032c58-5375-4b12-a083-06f04551a68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.682964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3b032c58-5375-4b12-a083-06f04551a68c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:37.684781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3b032c58-5375-4b12-a083-06f04551a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:37.736177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:37.776723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376140e8-5da4-436c-b8ca-44c6b6eb520c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:37.778256 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 83/177] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:37.779921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f855cfc-a7ee-4f27-b48e-a62bfe7eae21 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.780330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f855cfc-a7ee-4f27-b48e-a62bfe7eae21 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.780734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f855cfc-a7ee-4f27-b48e-a62bfe7eae21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.799393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7f855cfc-a7ee-4f27-b48e-a62bfe7eae21 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:37.800827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7f855cfc-a7ee-4f27-b48e-a62bfe7eae21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:37.836574 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c644aab-9e50-45ff-9c7d-b59d6bae3c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:37.837647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c644aab-9e50-45ff-9c7d-b59d6bae3c42 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.838014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c644aab-9e50-45ff-9c7d-b59d6bae3c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.838430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c644aab-9e50-45ff-9c7d-b59d6bae3c42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.852836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c644aab-9e50-45ff-9c7d-b59d6bae3c42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-7c644aab-9e50-45ff-9c7d-b59d6bae3c42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:37.861374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:37.872568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 95/178] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:37.873665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782e510-446b-4b04-bd23-130d4f9cf532 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:37.875056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782e510-446b-4b04-bd23-130d4f9cf532 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:37.875056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782e510-446b-4b04-bd23-130d4f9cf532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:37.891002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 84/179] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:37.892122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f855cfc-a7ee-4f27-b48e-a62bfe7eae21 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:37.897059 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5782e510-446b-4b04-bd23-130d4f9cf532 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:37.899440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5782e510-446b-4b04-bd23-130d4f9cf532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:37.930142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:38.024727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 96/180] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:38.026182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782e510-446b-4b04-bd23-130d4f9cf532 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:38.038477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:38.042957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f855cfc-a7ee-4f27-b48e-a62bfe7eae21 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:38.095486 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781373ad-5042-44b3-8b74-0d42fde33a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:38.096869 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781373ad-5042-44b3-8b74-0d42fde33a93 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:38.097166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781373ad-5042-44b3-8b74-0d42fde33a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:38.097446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781373ad-5042-44b3-8b74-0d42fde33a93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:38.183686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781373ad-5042-44b3-8b74-0d42fde33a93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:38.185123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781373ad-5042-44b3-8b74-0d42fde33a93 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:38.195558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782e510-446b-4b04-bd23-130d4f9cf532 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:38.255127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34700a6c-7a52-42f5-b907-8dcf718bc1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:38.256464 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34700a6c-7a52-42f5-b907-8dcf718bc1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:38.256984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34700a6c-7a52-42f5-b907-8dcf718bc1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:38.257482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34700a6c-7a52-42f5-b907-8dcf718bc1a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:38.277299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34700a6c-7a52-42f5-b907-8dcf718bc1a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-34700a6c-7a52-42f5-b907-8dcf718bc1a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:38.293797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:38.294841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:38.295691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:38.296401 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 97/181] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:10:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:38.297606 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34700a6c-7a52-42f5-b907-8dcf718bc1a7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:38.305331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:38.311284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 85/182] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:38.312416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781373ad-5042-44b3-8b74-0d42fde33a93 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:38.322237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:38.435348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781373ad-5042-44b3-8b74-0d42fde33a93 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:38.470903 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34700a6c-7a52-42f5-b907-8dcf718bc1a7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:38.478526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548821b-6ad3-473d-80ca-88be30705deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:38.479681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548821b-6ad3-473d-80ca-88be30705deb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:38.480143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548821b-6ad3-473d-80ca-88be30705deb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:38.480652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548821b-6ad3-473d-80ca-88be30705deb None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:38.481958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548821b-6ad3-473d-80ca-88be30705deb None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:38.482472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548821b-6ad3-473d-80ca-88be30705deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:38.485404 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548821b-6ad3-473d-80ca-88be30705deb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:38.498968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 86/183] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:38.500377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548821b-6ad3-473d-80ca-88be30705deb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:38.520045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8a65a-ecd1-40e0-a562-e60b2d0da607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:38.521475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8a65a-ecd1-40e0-a562-e60b2d0da607 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:38.521759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8a65a-ecd1-40e0-a562-e60b2d0da607 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:38.522030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8a65a-ecd1-40e0-a562-e60b2d0da607 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:38.547253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8a65a-ecd1-40e0-a562-e60b2d0da607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-25b8a65a-ecd1-40e0-a562-e60b2d0da607 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:38.556741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:38.559104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:38.560930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 98/184] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:10:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:38.563431 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8a65a-ecd1-40e0-a562-e60b2d0da607 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:38.584720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:38.641265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548821b-6ad3-473d-80ca-88be30705deb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:38.654657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:38.690441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d3ffa-5ef9-410c-93c3-cc65bd79a59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:38.691765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d3ffa-5ef9-410c-93c3-cc65bd79a59d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:38.692789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d3ffa-5ef9-410c-93c3-cc65bd79a59d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:38.693989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d3ffa-5ef9-410c-93c3-cc65bd79a59d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:38.699926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8a65a-ecd1-40e0-a562-e60b2d0da607 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:38.713626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d3ffa-5ef9-410c-93c3-cc65bd79a59d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-4b0d3ffa-5ef9-410c-93c3-cc65bd79a59d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:38.727243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:38.728881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:38.729860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 87/185] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:38.735432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d3ffa-5ef9-410c-93c3-cc65bd79a59d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:38.756657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e2cec-509c-408d-8c8e-ac977dffc5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:38.757938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e2cec-509c-408d-8c8e-ac977dffc5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:38.758498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e2cec-509c-408d-8c8e-ac977dffc5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:38.758807 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e2cec-509c-408d-8c8e-ac977dffc5b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:38.761811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e2cec-509c-408d-8c8e-ac977dffc5b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:38.764411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e2cec-509c-408d-8c8e-ac977dffc5b6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:38.767450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 99/186] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:38.768755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e2cec-509c-408d-8c8e-ac977dffc5b6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:38.868162 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d3ffa-5ef9-410c-93c3-cc65bd79a59d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:38.891578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e2cec-509c-408d-8c8e-ac977dffc5b6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:38.895765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:38.911616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03999a9-b103-4c60-8ec1-9f341ec1e831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:38.912460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03999a9-b103-4c60-8ec1-9f341ec1e831 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:38.912880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03999a9-b103-4c60-8ec1-9f341ec1e831 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:38.913177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03999a9-b103-4c60-8ec1-9f341ec1e831 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:38.913785 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03999a9-b103-4c60-8ec1-9f341ec1e831 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:38.915834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03999a9-b103-4c60-8ec1-9f341ec1e831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:38.915834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03999a9-b103-4c60-8ec1-9f341ec1e831 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:38.930913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e419b-e462-4ed5-a3a0-951570f865fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:38.931794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e419b-e462-4ed5-a3a0-951570f865fd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:38.932085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e419b-e462-4ed5-a3a0-951570f865fd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:38.932360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 88/187] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:38.932864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e419b-e462-4ed5-a3a0-951570f865fd None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:38.933226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03999a9-b103-4c60-8ec1-9f341ec1e831 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:38.934703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e419b-e462-4ed5-a3a0-951570f865fd None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:38.936615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e419b-e462-4ed5-a3a0-951570f865fd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:38.940895 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 100/188] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:38 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:38.940895 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e419b-e462-4ed5-a3a0-951570f865fd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:38.954639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:39.074840 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03999a9-b103-4c60-8ec1-9f341ec1e831 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:39.091409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e419b-e462-4ed5-a3a0-951570f865fd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:39.116893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb87ab-5774-4e3e-99aa-51a290f5f290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:39.117782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb87ab-5774-4e3e-99aa-51a290f5f290 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:39.118060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb87ab-5774-4e3e-99aa-51a290f5f290 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:39.118329 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb87ab-5774-4e3e-99aa-51a290f5f290 None admin] PATH_INFO: `/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:39.119630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb87ab-5774-4e3e-99aa-51a290f5f290 None admin] RBAC: Authorizing `identity:get_user(user_id=48574d0c8c5d4648b0a8c6d7d6d34287)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:39.121202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb87ab-5774-4e3e-99aa-51a290f5f290 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:39.128111 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 89/189] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:38 2022] GET /identity/v3/users/48574d0c8c5d4648b0a8c6d7d6d34287 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:39.128111 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb87ab-5774-4e3e-99aa-51a290f5f290 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:39.147748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ccd69-effc-4c53-8ed2-b0088f2fb8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:39.148437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ccd69-effc-4c53-8ed2-b0088f2fb8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:39.148723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ccd69-effc-4c53-8ed2-b0088f2fb8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:39.149007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ccd69-effc-4c53-8ed2-b0088f2fb8f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:39.149773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ccd69-effc-4c53-8ed2-b0088f2fb8f0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:39.150093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ccd69-effc-4c53-8ed2-b0088f2fb8f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:39.151545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ccd69-effc-4c53-8ed2-b0088f2fb8f0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:39.160301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 101/190] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:10:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:39.161889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ccd69-effc-4c53-8ed2-b0088f2fb8f0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:39.196992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:39.248350 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:39.280246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb87ab-5774-4e3e-99aa-51a290f5f290 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:39.286454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ccd69-effc-4c53-8ed2-b0088f2fb8f0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:39.325139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da37c0-53d0-4034-82ec-57bb194bc9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:39.325817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da37c0-53d0-4034-82ec-57bb194bc9da None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:39.326113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da37c0-53d0-4034-82ec-57bb194bc9da None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:39.326389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da37c0-53d0-4034-82ec-57bb194bc9da None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:39.326717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce162d2-89b5-4984-89f9-76caf5eda156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:39.327333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da37c0-53d0-4034-82ec-57bb194bc9da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:39.327458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce162d2-89b5-4984-89f9-76caf5eda156 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:39.327725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce162d2-89b5-4984-89f9-76caf5eda156 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:39.327966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce162d2-89b5-4984-89f9-76caf5eda156 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:39.328628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23da37c0-53d0-4034-82ec-57bb194bc9da None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:39.328764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce162d2-89b5-4984-89f9-76caf5eda156 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:39.329026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce162d2-89b5-4984-89f9-76caf5eda156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:39.330410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce162d2-89b5-4984-89f9-76caf5eda156 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:39.340646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 90/191] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:39.341516 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce162d2-89b5-4984-89f9-76caf5eda156 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:39.355707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 102/192] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:10:39 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:39.356683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da37c0-53d0-4034-82ec-57bb194bc9da None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:39.488423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce162d2-89b5-4984-89f9-76caf5eda156 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:39.505338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:39.522019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da37c0-53d0-4034-82ec-57bb194bc9da None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:39.531261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a1295-8c13-482a-bafe-7930c502bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:39.532831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a1295-8c13-482a-bafe-7930c502bc37 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:39.532831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a1295-8c13-482a-bafe-7930c502bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:39.532831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a1295-8c13-482a-bafe-7930c502bc37 None admin] PATH_INFO: `/v3/users/e8f24e7c37e54fa788135577d7ace1ad` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:39.551955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a1295-8c13-482a-bafe-7930c502bc37 None admin] RBAC: Authorizing `identity:get_user(user_id=e8f24e7c37e54fa788135577d7ace1ad)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:39.554018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a1295-8c13-482a-bafe-7930c502bc37 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:39.556143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 91/193] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:39 2022] GET /identity/v3/users/e8f24e7c37e54fa788135577d7ace1ad => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:39.560878 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a1295-8c13-482a-bafe-7930c502bc37 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:39.562078 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:39.576508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6485cca-347c-48c1-bfc4-73c265e64461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:39.577203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6485cca-347c-48c1-bfc4-73c265e64461 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:39.577737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6485cca-347c-48c1-bfc4-73c265e64461 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:39.577971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6485cca-347c-48c1-bfc4-73c265e64461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:39.578995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6485cca-347c-48c1-bfc4-73c265e64461 None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=5902af097cc346559cbefdb217f7394c, user.id=c5ec92316a2c452dab52aa49d7256f2f) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:39.579281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6485cca-347c-48c1-bfc4-73c265e64461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:39.580891 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6485cca-347c-48c1-bfc4-73c265e64461 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:39.588493 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 103/194] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:39 2022] GET /identity/v3/role_assignments?user.id=c5ec92316a2c452dab52aa49d7256f2f&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:39.589234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6485cca-347c-48c1-bfc4-73c265e64461 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:39.740757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a1295-8c13-482a-bafe-7930c502bc37 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:39.741261 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6485cca-347c-48c1-bfc4-73c265e64461 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:39.801823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656f457-774d-4d5c-a3de-e04e3bfe3dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:39.802573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656f457-774d-4d5c-a3de-e04e3bfe3dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:39.802886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656f457-774d-4d5c-a3de-e04e3bfe3dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:39.803163 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656f457-774d-4d5c-a3de-e04e3bfe3dd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:39.807507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ec695-cbfa-43f3-b81a-2cacb70803f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:39.808838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ec695-cbfa-43f3-b81a-2cacb70803f0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:39.809380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ec695-cbfa-43f3-b81a-2cacb70803f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:39.809969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ec695-cbfa-43f3-b81a-2cacb70803f0 None admin] PATH_INFO: `/v3/users/e8f24e7c37e54fa788135577d7ace1ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:39.811852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ec695-cbfa-43f3-b81a-2cacb70803f0 None admin] RBAC: Authorizing `identity:get_user(user_id=e8f24e7c37e54fa788135577d7ace1ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:39.813833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7ec695-cbfa-43f3-b81a-2cacb70803f0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:39.816143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 104/195] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:39 2022] GET /identity/v3/users/e8f24e7c37e54fa788135577d7ace1ad => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:39.817879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ec695-cbfa-43f3-b81a-2cacb70803f0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:39.886987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:39.903747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:39.950725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ec695-cbfa-43f3-b81a-2cacb70803f0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:39.954926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656f457-774d-4d5c-a3de-e04e3bfe3dd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:39.957284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656f457-774d-4d5c-a3de-e04e3bfe3dd7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:40.012878 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542049e-1259-47cd-a9f7-51595f1b2581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:40.014413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542049e-1259-47cd-a9f7-51595f1b2581 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:40.014830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542049e-1259-47cd-a9f7-51595f1b2581 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:40.015340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542049e-1259-47cd-a9f7-51595f1b2581 None admin] PATH_INFO: `/v3/users/ff084582e0db4ebe8c66948ef96a56da` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:40.026235 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542049e-1259-47cd-a9f7-51595f1b2581 None admin] RBAC: Authorizing `identity:get_user(user_id=ff084582e0db4ebe8c66948ef96a56da)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:40.029302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c542049e-1259-47cd-a9f7-51595f1b2581 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:40.032065 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 105/196] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:39 2022] GET /identity/v3/users/ff084582e0db4ebe8c66948ef96a56da => generated 289 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:40.033564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542049e-1259-47cd-a9f7-51595f1b2581 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:40.107264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 92/197] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:40.108442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656f457-774d-4d5c-a3de-e04e3bfe3dd7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:40.203533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542049e-1259-47cd-a9f7-51595f1b2581 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:40.214364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:40.246378 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:40.253121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656f457-774d-4d5c-a3de-e04e3bfe3dd7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:40.264996 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81262073-a372-49d3-b420-ae4e6aef53b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:40.266907 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262073-a372-49d3-b420-ae4e6aef53b8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:40.267325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262073-a372-49d3-b420-ae4e6aef53b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:40.268896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81262073-a372-49d3-b420-ae4e6aef53b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:40.305863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6445e8-a236-4eb3-833b-9de7ed486eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:40.306566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6445e8-a236-4eb3-833b-9de7ed486eae None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:40.306890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6445e8-a236-4eb3-833b-9de7ed486eae None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:40.307664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6445e8-a236-4eb3-833b-9de7ed486eae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:40.362720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81262073-a372-49d3-b420-ae4e6aef53b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:40.366088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81262073-a372-49d3-b420-ae4e6aef53b8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:40.408915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6445e8-a236-4eb3-833b-9de7ed486eae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:40.410483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6445e8-a236-4eb3-833b-9de7ed486eae None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:40.513628 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:40.531101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 106/198] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:40.532355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81262073-a372-49d3-b420-ae4e6aef53b8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:40.556973 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 93/199] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:40.557889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6445e8-a236-4eb3-833b-9de7ed486eae None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:40.570319 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:40.659418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81262073-a372-49d3-b420-ae4e6aef53b8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:40.696894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6445e8-a236-4eb3-833b-9de7ed486eae None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:40.704728 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edcf4e-9adf-4984-b314-95507df53a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:40.706358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edcf4e-9adf-4984-b314-95507df53a25 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:40.707120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edcf4e-9adf-4984-b314-95507df53a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:40.708461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edcf4e-9adf-4984-b314-95507df53a25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:40.748931 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00375a-771a-4993-a128-b26bbd61ecba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:40.750405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00375a-771a-4993-a128-b26bbd61ecba None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:40.750946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00375a-771a-4993-a128-b26bbd61ecba None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:40.751426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00375a-771a-4993-a128-b26bbd61ecba None admin] PATH_INFO: `/v3/projects/663aac864cb941cc8165cc02ae504cf6` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:40.752994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00375a-771a-4993-a128-b26bbd61ecba None admin] RBAC: Authorizing `identity:get_project(project_id=663aac864cb941cc8165cc02ae504cf6)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:40.755394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa00375a-771a-4993-a128-b26bbd61ecba None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:40.758059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 94/200] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:40 2022] GET /identity/v3/projects/663aac864cb941cc8165cc02ae504cf6 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:40.759115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00375a-771a-4993-a128-b26bbd61ecba None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:40.812756 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:40.815505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edcf4e-9adf-4984-b314-95507df53a25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:40.819491 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edcf4e-9adf-4984-b314-95507df53a25 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:40.860945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:40.920870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa00375a-771a-4993-a128-b26bbd61ecba None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:40.968669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 107/201] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:40.973628 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fcdc5-325c-429c-8ac5-954d4e6d58f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:40.974400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fcdc5-325c-429c-8ac5-954d4e6d58f6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:40.974920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fcdc5-325c-429c-8ac5-954d4e6d58f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:40.975267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fcdc5-325c-429c-8ac5-954d4e6d58f6 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:40.975650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edcf4e-9adf-4984-b314-95507df53a25 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:40.976953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fcdc5-325c-429c-8ac5-954d4e6d58f6 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:40.979125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fcdc5-325c-429c-8ac5-954d4e6d58f6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:40.982210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 95/202] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:40 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:40.983203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fcdc5-325c-429c-8ac5-954d4e6d58f6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:41.124889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edcf4e-9adf-4984-b314-95507df53a25 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:41.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:41.162755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fcdc5-325c-429c-8ac5-954d4e6d58f6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:41.173552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:41.180642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9614016-dd70-4ace-b564-ec0c53a6236d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:41.182317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614016-dd70-4ace-b564-ec0c53a6236d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:41.182834 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614016-dd70-4ace-b564-ec0c53a6236d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:41.183351 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9614016-dd70-4ace-b564-ec0c53a6236d None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:41.185473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9614016-dd70-4ace-b564-ec0c53a6236d None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:41.187699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9614016-dd70-4ace-b564-ec0c53a6236d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:41.190666 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 108/203] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:40 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:41.194303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9614016-dd70-4ace-b564-ec0c53a6236d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:41.220228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba6af4-e1cc-4fe4-855e-447683fb8f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:41.222118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba6af4-e1cc-4fe4-855e-447683fb8f84 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:41.224402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba6af4-e1cc-4fe4-855e-447683fb8f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:41.225349 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba6af4-e1cc-4fe4-855e-447683fb8f84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:41.227467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba6af4-e1cc-4fe4-855e-447683fb8f84 None admin] RBAC: Adding query filter params (role.id=ef48ab9c5b844be59351a260130d3071, scope.project.id=663aac864cb941cc8165cc02ae504cf6, user.id=48574d0c8c5d4648b0a8c6d7d6d34287) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:41.228260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba6af4-e1cc-4fe4-855e-447683fb8f84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:41.231438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba6af4-e1cc-4fe4-855e-447683fb8f84 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:41.243560 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 96/204] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:40 2022] GET /identity/v3/role_assignments?user.id=48574d0c8c5d4648b0a8c6d7d6d34287&scope.project.id=663aac864cb941cc8165cc02ae504cf6&role.id=ef48ab9c5b844be59351a260130d3071 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:41.245031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba6af4-e1cc-4fe4-855e-447683fb8f84 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:41.453492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba6af4-e1cc-4fe4-855e-447683fb8f84 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:41.469671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9614016-dd70-4ace-b564-ec0c53a6236d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:41.496059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e1cce-0694-4799-89e8-bcaef93a77f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:41.496747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e1cce-0694-4799-89e8-bcaef93a77f4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:41.497044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e1cce-0694-4799-89e8-bcaef93a77f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:41.497335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e1cce-0694-4799-89e8-bcaef93a77f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:41.498317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e1cce-0694-4799-89e8-bcaef93a77f4 None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=366a6c5b1cb444698289e32b2b46918d, user.id=e8f24e7c37e54fa788135577d7ace1ad) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:41.498701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e1cce-0694-4799-89e8-bcaef93a77f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:41.500460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e1cce-0694-4799-89e8-bcaef93a77f4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:41.507930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 97/205] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:41 2022] GET /identity/v3/role_assignments?user.id=e8f24e7c37e54fa788135577d7ace1ad&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:41.508816 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e1cce-0694-4799-89e8-bcaef93a77f4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:41.556604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5883a40-dc1a-4013-9251-63518d31ecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:41.557659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5883a40-dc1a-4013-9251-63518d31ecd3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:41.558048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5883a40-dc1a-4013-9251-63518d31ecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:41.558485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5883a40-dc1a-4013-9251-63518d31ecd3 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:41.563699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5883a40-dc1a-4013-9251-63518d31ecd3 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:41.565341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5883a40-dc1a-4013-9251-63518d31ecd3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:41.568287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 109/206] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:41 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:41.573689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:41.574313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5883a40-dc1a-4013-9251-63518d31ecd3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:41.668907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e1cce-0694-4799-89e8-bcaef93a77f4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:41.682300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:41.726313 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f6cbf-8252-4cce-a603-b0b013cb108f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:41.727004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f6cbf-8252-4cce-a603-b0b013cb108f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:41.727335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f6cbf-8252-4cce-a603-b0b013cb108f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:41.727581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f6cbf-8252-4cce-a603-b0b013cb108f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:41.728532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f6cbf-8252-4cce-a603-b0b013cb108f None admin] RBAC: Adding query filter params (role.id=1d508b4effb546039e87dd273933af16, scope.project.id=366a6c5b1cb444698289e32b2b46918d, user.id=e8f24e7c37e54fa788135577d7ace1ad) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:41.728853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f6cbf-8252-4cce-a603-b0b013cb108f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:41.730680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f6cbf-8252-4cce-a603-b0b013cb108f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:41.738374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 98/207] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:41 2022] GET /identity/v3/role_assignments?user.id=e8f24e7c37e54fa788135577d7ace1ad&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=1d508b4effb546039e87dd273933af16 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:41.762851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5883a40-dc1a-4013-9251-63518d31ecd3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:41.819083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59094f38-b7b5-46ba-b3b2-ad1ddfd96cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:41.819795 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59094f38-b7b5-46ba-b3b2-ad1ddfd96cdb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:41.820080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59094f38-b7b5-46ba-b3b2-ad1ddfd96cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:41.820362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59094f38-b7b5-46ba-b3b2-ad1ddfd96cdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:41.821409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59094f38-b7b5-46ba-b3b2-ad1ddfd96cdb None admin] RBAC: Adding query filter params (role.id=53b551bc12e84268991ca46bce5da0ec, scope.project.id=5902af097cc346559cbefdb217f7394c, user.id=ff084582e0db4ebe8c66948ef96a56da) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:41.821725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59094f38-b7b5-46ba-b3b2-ad1ddfd96cdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:41.823331 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59094f38-b7b5-46ba-b3b2-ad1ddfd96cdb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:41.833117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 110/208] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:41 2022] GET /identity/v3/role_assignments?user.id=ff084582e0db4ebe8c66948ef96a56da&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=53b551bc12e84268991ca46bce5da0ec => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:42.158956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b756dd-055e-41cd-ada6-5292c2d588c5 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:42.159378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b756dd-055e-41cd-ada6-5292c2d588c5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:42.159531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b756dd-055e-41cd-ada6-5292c2d588c5 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:42.162009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 99/209] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:10:42.168844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ec935-f2cf-4530-845e-0f90f0c31a68 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:42.169174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ec935-f2cf-4530-845e-0f90f0c31a68 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:42.169410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ec935-f2cf-4530-845e-0f90f0c31a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:42.194085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-033ec935-f2cf-4530-845e-0f90f0c31a68 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:42.195357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-033ec935-f2cf-4530-845e-0f90f0c31a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:42.313570 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 111/210] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:42.337581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b756dd-055e-41cd-ada6-5292c2d588c5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:42.485890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:42.489608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b756dd-055e-41cd-ada6-5292c2d588c5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:42.500421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd9396-7abc-415f-86b0-9c632381ece1 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:42.501948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd9396-7abc-415f-86b0-9c632381ece1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:42.502804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd9396-7abc-415f-86b0-9c632381ece1 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:42.507500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 112/211] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:42.516681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a182333-6108-408d-94d1-21e26bfe33b4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:42.517635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a182333-6108-408d-94d1-21e26bfe33b4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:42.518822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a182333-6108-408d-94d1-21e26bfe33b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:42.534044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8449a-6379-4989-a217-481df24e419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:42.534916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8449a-6379-4989-a217-481df24e419b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:42.535209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8449a-6379-4989-a217-481df24e419b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:42.535473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e8449a-6379-4989-a217-481df24e419b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:42.549826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e8449a-6379-4989-a217-481df24e419b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:42.552357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4a182333-6108-408d-94d1-21e26bfe33b4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:42.554824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4a182333-6108-408d-94d1-21e26bfe33b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d4e8449a-6379-4989-a217-481df24e419b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:42.559562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:42.561340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:42.563003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 100/212] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:42.568906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8449a-6379-4989-a217-481df24e419b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:42.668073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:42.691095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e8449a-6379-4989-a217-481df24e419b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:42.714980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 113/213] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:42.733035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6c103-c9b4-4d98-8f7d-a6167f082e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:42.733891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c103-c9b4-4d98-8f7d-a6167f082e14 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:42.734189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c103-c9b4-4d98-8f7d-a6167f082e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:42.734564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6c103-c9b4-4d98-8f7d-a6167f082e14 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:42.735725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6c103-c9b4-4d98-8f7d-a6167f082e14 None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:42.736043 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6c103-c9b4-4d98-8f7d-a6167f082e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:42.737460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6c103-c9b4-4d98-8f7d-a6167f082e14 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:42.748752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 101/214] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:42.755051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a182333-6108-408d-94d1-21e26bfe33b4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:42.765988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6c103-c9b4-4d98-8f7d-a6167f082e14 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:42.826825 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:42.917785 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6c103-c9b4-4d98-8f7d-a6167f082e14 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:42.951307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a182333-6108-408d-94d1-21e26bfe33b4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:42.979580 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9854cb0-d96b-4fc8-9d93-ae1b5ef79982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:42.981114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9854cb0-d96b-4fc8-9d93-ae1b5ef79982 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:42.981415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9854cb0-d96b-4fc8-9d93-ae1b5ef79982 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:42.981838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9854cb0-d96b-4fc8-9d93-ae1b5ef79982 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:42.983978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9854cb0-d96b-4fc8-9d93-ae1b5ef79982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:42.986887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9854cb0-d96b-4fc8-9d93-ae1b5ef79982 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:42.996527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 102/215] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:42.998704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9854cb0-d96b-4fc8-9d93-ae1b5ef79982 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:43.034913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa3c22-0a24-4999-8164-cbb79c9b64de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:43.037380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa3c22-0a24-4999-8164-cbb79c9b64de None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:43.038283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa3c22-0a24-4999-8164-cbb79c9b64de None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:43.039289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa3c22-0a24-4999-8164-cbb79c9b64de None admin] PATH_INFO: `/v3/users/f9c63e2687f34df5ad7c08a5002b2429` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:43.066413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa3c22-0a24-4999-8164-cbb79c9b64de None admin] RBAC: Authorizing `identity:get_user(user_id=f9c63e2687f34df5ad7c08a5002b2429)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:43.069766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa3c22-0a24-4999-8164-cbb79c9b64de None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:43.073644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 114/216] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:42 2022] GET /identity/v3/users/f9c63e2687f34df5ad7c08a5002b2429 => generated 297 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:43.080301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa3c22-0a24-4999-8164-cbb79c9b64de None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:43.154930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:43.167689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9854cb0-d96b-4fc8-9d93-ae1b5ef79982 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:43.180680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:43.212688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e6160-fc93-42de-a3f2-9be3513e369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:43.213481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e6160-fc93-42de-a3f2-9be3513e369e None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:43.213751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e6160-fc93-42de-a3f2-9be3513e369e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:43.214029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e6160-fc93-42de-a3f2-9be3513e369e None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:43.214809 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e6160-fc93-42de-a3f2-9be3513e369e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:43.216344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e6160-fc93-42de-a3f2-9be3513e369e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:43.246655 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 103/217] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:10:42 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:43.277334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa3c22-0a24-4999-8164-cbb79c9b64de None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:43.324726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d2f9c-11da-4bcb-929c-5de7882b1f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:43.325691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2f9c-11da-4bcb-929c-5de7882b1f40 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:43.326122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2f9c-11da-4bcb-929c-5de7882b1f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:43.326541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d2f9c-11da-4bcb-929c-5de7882b1f40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:43.429613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d2f9c-11da-4bcb-929c-5de7882b1f40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:43.431170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6d2f9c-11da-4bcb-929c-5de7882b1f40 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:43.488173 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:43.566613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 115/218] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:43.575544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e6160-fc93-42de-a3f2-9be3513e369e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:43.721796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e6160-fc93-42de-a3f2-9be3513e369e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:43.772944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3205525-11b6-44b6-a15d-989d9c1235dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:43.772944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3205525-11b6-44b6-a15d-989d9c1235dc None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:43.772944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3205525-11b6-44b6-a15d-989d9c1235dc None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:43.772944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3205525-11b6-44b6-a15d-989d9c1235dc None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:43.773791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3205525-11b6-44b6-a15d-989d9c1235dc None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:43.775492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3205525-11b6-44b6-a15d-989d9c1235dc None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:43.777615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 104/219] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:43 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:43.786702 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d2f9c-11da-4bcb-929c-5de7882b1f40 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:44.003243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d2f9c-11da-4bcb-929c-5de7882b1f40 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:44.092599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb36c8-87d3-49b8-a900-6cefbaf95e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:44.093748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb36c8-87d3-49b8-a900-6cefbaf95e2d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:44.094203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb36c8-87d3-49b8-a900-6cefbaf95e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:44.094673 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb36c8-87d3-49b8-a900-6cefbaf95e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:44.096221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb36c8-87d3-49b8-a900-6cefbaf95e2d None admin] RBAC: Adding query filter params (role.id=ef48ab9c5b844be59351a260130d3071, scope.project.id=366a6c5b1cb444698289e32b2b46918d, user.id=f9c63e2687f34df5ad7c08a5002b2429) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:44.096731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb36c8-87d3-49b8-a900-6cefbaf95e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:44.100195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb36c8-87d3-49b8-a900-6cefbaf95e2d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:44.117487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 116/220] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:43 2022] GET /identity/v3/role_assignments?user.id=f9c63e2687f34df5ad7c08a5002b2429&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=ef48ab9c5b844be59351a260130d3071 => generated 259 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:44.465038 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c848be7-b356-412e-b738-3fed585a2add None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:44.465725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c848be7-b356-412e-b738-3fed585a2add None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:44.466213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c848be7-b356-412e-b738-3fed585a2add None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:44.468272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 105/221] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:44.474677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2474cc-3c21-4ce7-b9b5-a2e02031fd95 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:44.475314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2474cc-3c21-4ce7-b9b5-a2e02031fd95 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:44.475787 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2474cc-3c21-4ce7-b9b5-a2e02031fd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:44.495531 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:44.507568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2d2474cc-3c21-4ce7-b9b5-a2e02031fd95 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:44.509203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2d2474cc-3c21-4ce7-b9b5-a2e02031fd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:44.611864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 117/222] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:44.648359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c848be7-b356-412e-b738-3fed585a2add None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:44.689067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:44.812391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d6a2f-438b-409d-93e9-86fbc59fc3ff None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:44.812871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d6a2f-438b-409d-93e9-86fbc59fc3ff None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:44.813024 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d6a2f-438b-409d-93e9-86fbc59fc3ff None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:44.814425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 118/223] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:44.838673 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c020b-975a-4265-9f11-636a29caee2e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:44.839096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c020b-975a-4265-9f11-636a29caee2e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:44.839270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c020b-975a-4265-9f11-636a29caee2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:44.872146 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-848c020b-975a-4265-9f11-636a29caee2e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:44.873391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-848c020b-975a-4265-9f11-636a29caee2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:44.897758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c848be7-b356-412e-b738-3fed585a2add None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:44.971053 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c16d3f-c049-441c-a1cc-97ad32d2c41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:44.971883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c16d3f-c049-441c-a1cc-97ad32d2c41a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:44.972175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c16d3f-c049-441c-a1cc-97ad32d2c41a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:44.972448 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c16d3f-c049-441c-a1cc-97ad32d2c41a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:45.000514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c16d3f-c049-441c-a1cc-97ad32d2c41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-53c16d3f-c049-441c-a1cc-97ad32d2c41a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:45.008921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:45.010734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:45.012362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:45.013578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 106/224] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:10:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:45.027348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c16d3f-c049-441c-a1cc-97ad32d2c41a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:45.062139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 119/225] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:45.131324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c020b-975a-4265-9f11-636a29caee2e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:45.163183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:45.189444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c16d3f-c049-441c-a1cc-97ad32d2c41a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:45.229223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:45.237585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8244c-fc5d-4781-b5d1-583e10ee76ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:45.238388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8244c-fc5d-4781-b5d1-583e10ee76ca None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:45.238790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8244c-fc5d-4781-b5d1-583e10ee76ca None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:45.239220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8244c-fc5d-4781-b5d1-583e10ee76ca None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:45.240172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8244c-fc5d-4781-b5d1-583e10ee76ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:45.240831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8244c-fc5d-4781-b5d1-583e10ee76ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:45.242473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8244c-fc5d-4781-b5d1-583e10ee76ca None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:45.252823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 107/226] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:45.259594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8244c-fc5d-4781-b5d1-583e10ee76ca None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:45.303910 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c020b-975a-4265-9f11-636a29caee2e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:45.377100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf4c88-6504-4122-b92e-064e0db6b59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:45.378054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf4c88-6504-4122-b92e-064e0db6b59f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:45.378336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf4c88-6504-4122-b92e-064e0db6b59f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:45.382719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf4c88-6504-4122-b92e-064e0db6b59f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:45.409803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebf4c88-6504-4122-b92e-064e0db6b59f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:45.424070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8244c-fc5d-4781-b5d1-583e10ee76ca None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-9ebf4c88-6504-4122-b92e-064e0db6b59f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:45.429948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:45.431680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:45.433141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:45.437774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 120/227] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:10:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:10:45.445644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf4c88-6504-4122-b92e-064e0db6b59f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:45.488398 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303920c-226f-41ca-b80d-55190a0af2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:45.489469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303920c-226f-41ca-b80d-55190a0af2ab None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:45.489875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303920c-226f-41ca-b80d-55190a0af2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:45.494637 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303920c-226f-41ca-b80d-55190a0af2ab None admin] PATH_INFO: `/v3/users/e8f24e7c37e54fa788135577d7ace1ad` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:45.494637 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303920c-226f-41ca-b80d-55190a0af2ab None admin] RBAC: Authorizing `identity:get_user(user_id=e8f24e7c37e54fa788135577d7ace1ad)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:45.497255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303920c-226f-41ca-b80d-55190a0af2ab None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:45.507909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 108/228] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:45 2022] GET /identity/v3/users/e8f24e7c37e54fa788135577d7ace1ad => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:45.517403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303920c-226f-41ca-b80d-55190a0af2ab None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:45.523065 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:45.737534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf4c88-6504-4122-b92e-064e0db6b59f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:45.779358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303920c-226f-41ca-b80d-55190a0af2ab None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:45.809004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:45.823034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1d391-15e5-40d6-ad6d-a41a070cdc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:45.824330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1d391-15e5-40d6-ad6d-a41a070cdc8b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:45.824627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1d391-15e5-40d6-ad6d-a41a070cdc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:45.824914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1d391-15e5-40d6-ad6d-a41a070cdc8b None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:45.825702 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1d391-15e5-40d6-ad6d-a41a070cdc8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:45.826021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1d391-15e5-40d6-ad6d-a41a070cdc8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:45.827624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d1d391-15e5-40d6-ad6d-a41a070cdc8b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:45.839105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 121/229] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:10:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:45.845587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1d391-15e5-40d6-ad6d-a41a070cdc8b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:45.955782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf8172-21b2-4689-bc28-988f178f4247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:45.957947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8172-21b2-4689-bc28-988f178f4247 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:45.958466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8172-21b2-4689-bc28-988f178f4247 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:45.959309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8172-21b2-4689-bc28-988f178f4247 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:45.968041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf8172-21b2-4689-bc28-988f178f4247 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:45.970858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf8172-21b2-4689-bc28-988f178f4247 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:45.974993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 109/230] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:45 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:45.984312 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf8172-21b2-4689-bc28-988f178f4247 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:45.993565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1d391-15e5-40d6-ad6d-a41a070cdc8b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:46.038917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4aff5-9137-4955-ac05-f5238871902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:46.039970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4aff5-9137-4955-ac05-f5238871902a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:46.040268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4aff5-9137-4955-ac05-f5238871902a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:46.040525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4aff5-9137-4955-ac05-f5238871902a None admin] PATH_INFO: `/v3/users/e8f24e7c37e54fa788135577d7ace1ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:46.042554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4aff5-9137-4955-ac05-f5238871902a None admin] RBAC: Authorizing `identity:get_user(user_id=e8f24e7c37e54fa788135577d7ace1ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:46.044404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4aff5-9137-4955-ac05-f5238871902a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:46.046517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 122/231] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:45 2022] GET /identity/v3/users/e8f24e7c37e54fa788135577d7ace1ad => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:46.053522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4aff5-9137-4955-ac05-f5238871902a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:46.128346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:46.183823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4aff5-9137-4955-ac05-f5238871902a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:46.229984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f318f-6195-4931-9099-5545785d27aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:46.230928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f318f-6195-4931-9099-5545785d27aa None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:46.231219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f318f-6195-4931-9099-5545785d27aa None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:46.231487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f318f-6195-4931-9099-5545785d27aa None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:46.232858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f318f-6195-4931-9099-5545785d27aa None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:46.234495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f318f-6195-4931-9099-5545785d27aa None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:46.237013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 123/232] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:46 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:46.243852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f318f-6195-4931-9099-5545785d27aa None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:46.340405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:46.389393 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f318f-6195-4931-9099-5545785d27aa None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:46.409187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf8172-21b2-4689-bc28-988f178f4247 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:46.417734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:46.437025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38a4f0-e9c8-44e7-b032-85c6d090585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:46.438024 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38a4f0-e9c8-44e7-b032-85c6d090585b None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:46.438320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38a4f0-e9c8-44e7-b032-85c6d090585b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:46.438623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38a4f0-e9c8-44e7-b032-85c6d090585b None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d/users/e8f24e7c37e54fa788135577d7ace1ad/roles/1d508b4effb546039e87dd273933af16` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:46.443293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38a4f0-e9c8-44e7-b032-85c6d090585b None admin] RBAC: Authorizing `identity:create_grant(project_id=366a6c5b1cb444698289e32b2b46918d, user_id=e8f24e7c37e54fa788135577d7ace1ad, role_id=1d508b4effb546039e87dd273933af16)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:46.448045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38a4f0-e9c8-44e7-b032-85c6d090585b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:46.475800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 124/233] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:10:46 2022] PUT /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d/users/e8f24e7c37e54fa788135577d7ace1ad/roles/1d508b4effb546039e87dd273933af16 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:10:46.496135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48926820-ffcc-4690-a887-d98f7162cffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:46.497008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48926820-ffcc-4690-a887-d98f7162cffd None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:46.497352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48926820-ffcc-4690-a887-d98f7162cffd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:46.497724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48926820-ffcc-4690-a887-d98f7162cffd None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d/users/e8f24e7c37e54fa788135577d7ace1ad/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:46.502987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48926820-ffcc-4690-a887-d98f7162cffd None admin] RBAC: Authorizing `identity:create_grant(project_id=366a6c5b1cb444698289e32b2b46918d, user_id=e8f24e7c37e54fa788135577d7ace1ad, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:46.505283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48926820-ffcc-4690-a887-d98f7162cffd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:46.533303 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 110/234] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:10:45 2022] PUT /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d/users/e8f24e7c37e54fa788135577d7ace1ad/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:10:46.970622 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafee14f-7a00-45ee-bcfc-43774e9b3ff4 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:46.971610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafee14f-7a00-45ee-bcfc-43774e9b3ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:46.971610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafee14f-7a00-45ee-bcfc-43774e9b3ff4 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:46.973142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 125/235] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:46.979499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61000140-9036-4817-8d88-7771a144f75d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:46.980305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61000140-9036-4817-8d88-7771a144f75d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:46.980305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61000140-9036-4817-8d88-7771a144f75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:47.007863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-61000140-9036-4817-8d88-7771a144f75d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:47.009246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-61000140-9036-4817-8d88-7771a144f75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:47.125275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 111/236] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:47.172267 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafee14f-7a00-45ee-bcfc-43774e9b3ff4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:47.336590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafee14f-7a00-45ee-bcfc-43774e9b3ff4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:47.395134 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d4453-416e-489f-af4e-3c6e24f32dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:47.396468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d4453-416e-489f-af4e-3c6e24f32dab None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:47.397055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d4453-416e-489f-af4e-3c6e24f32dab None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:47.397402 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d4453-416e-489f-af4e-3c6e24f32dab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:47.421091 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d4453-416e-489f-af4e-3c6e24f32dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-6b8d4453-416e-489f-af4e-3c6e24f32dab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:47.430271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:47.432209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:47.433870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:47.436432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 126/237] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:47.446455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61000140-9036-4817-8d88-7771a144f75d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:47.466552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160ade7-f0ed-4263-ae40-a1460754ef53 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:47.467019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160ade7-f0ed-4263-ae40-a1460754ef53 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:47.467291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e160ade7-f0ed-4263-ae40-a1460754ef53 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:47.468923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 127/238] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:47.476813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafc758-4f85-4d32-97cc-f11d354e3922 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:47.477252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafc758-4f85-4d32-97cc-f11d354e3922 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:47.477511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafc758-4f85-4d32-97cc-f11d354e3922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:47.499222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6cafc758-4f85-4d32-97cc-f11d354e3922 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:47.501065 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6cafc758-4f85-4d32-97cc-f11d354e3922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:47.559052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:47.579178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:47.600068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 128/239] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:47.602720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423307d7-6ef9-4b57-bcc6-adb5e2cdc15e None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:47.603068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423307d7-6ef9-4b57-bcc6-adb5e2cdc15e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:47.603315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423307d7-6ef9-4b57-bcc6-adb5e2cdc15e None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:47.604779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 129/240] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:47.615496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e81b-136a-4963-b793-5c79f1c70f67 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:47.615774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e81b-136a-4963-b793-5c79f1c70f67 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:47.616057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e81b-136a-4963-b793-5c79f1c70f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:47.646941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9327e81b-136a-4963-b793-5c79f1c70f67 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:47.648264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9327e81b-136a-4963-b793-5c79f1c70f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:47.654446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61000140-9036-4817-8d88-7771a144f75d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:47.703621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf902b4-54f8-4b49-9dd4-288b799f12fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:47.704327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf902b4-54f8-4b49-9dd4-288b799f12fb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:47.705042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf902b4-54f8-4b49-9dd4-288b799f12fb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:47.705356 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf902b4-54f8-4b49-9dd4-288b799f12fb None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:47.706168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf902b4-54f8-4b49-9dd4-288b799f12fb None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:47.706489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf902b4-54f8-4b49-9dd4-288b799f12fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:47.709828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf902b4-54f8-4b49-9dd4-288b799f12fb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:47.729373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 112/241] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:47.730505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf902b4-54f8-4b49-9dd4-288b799f12fb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:47.745868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 130/242] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:47.746959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327e81b-136a-4963-b793-5c79f1c70f67 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:47.892813 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf902b4-54f8-4b49-9dd4-288b799f12fb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:47.926700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:47.945977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbab881-973b-4e21-9023-06f6b610341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:47.948671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbab881-973b-4e21-9023-06f6b610341b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:47.949506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbab881-973b-4e21-9023-06f6b610341b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:47.950283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbab881-973b-4e21-9023-06f6b610341b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:47.967223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327e81b-136a-4963-b793-5c79f1c70f67 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:47.971736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:47.980601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbab881-973b-4e21-9023-06f6b610341b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ebbab881-973b-4e21-9023-06f6b610341b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:47.997385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:47.998916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:48.001117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:48.003033 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 113/243] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:48.004148 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbab881-973b-4e21-9023-06f6b610341b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:48.047330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809972d-74ff-4219-8538-6fac2ef270a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:48.050396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809972d-74ff-4219-8538-6fac2ef270a8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:48.051604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809972d-74ff-4219-8538-6fac2ef270a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:48.052413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809972d-74ff-4219-8538-6fac2ef270a8 None admin] PATH_INFO: `/v3/users/f9c63e2687f34df5ad7c08a5002b2429` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:48.055476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809972d-74ff-4219-8538-6fac2ef270a8 None admin] RBAC: Authorizing `identity:get_user(user_id=f9c63e2687f34df5ad7c08a5002b2429)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:48.057127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809972d-74ff-4219-8538-6fac2ef270a8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:48.062621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 131/244] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:47 2022] GET /identity/v3/users/f9c63e2687f34df5ad7c08a5002b2429 => generated 297 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:48.062621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809972d-74ff-4219-8538-6fac2ef270a8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:48.189758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809972d-74ff-4219-8538-6fac2ef270a8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:48.191061 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbab881-973b-4e21-9023-06f6b610341b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:48.247897 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c81c-c69b-488e-8855-324b63ce0e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:48.250018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c81c-c69b-488e-8855-324b63ce0e6b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:48.250862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c81c-c69b-488e-8855-324b63ce0e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:48.251656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4c81c-c69b-488e-8855-324b63ce0e6b None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:48.253989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4c81c-c69b-488e-8855-324b63ce0e6b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:48.254862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4c81c-c69b-488e-8855-324b63ce0e6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:48.255731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681ec31-06e4-4b74-a37b-8555278c3a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:48.256610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4c81c-c69b-488e-8855-324b63ce0e6b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:48.257114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681ec31-06e4-4b74-a37b-8555278c3a71 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:48.257589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681ec31-06e4-4b74-a37b-8555278c3a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:48.258081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4681ec31-06e4-4b74-a37b-8555278c3a71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:48.260466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681ec31-06e4-4b74-a37b-8555278c3a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:48.262671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4681ec31-06e4-4b74-a37b-8555278c3a71 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:48.265876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 114/245] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:48.267215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681ec31-06e4-4b74-a37b-8555278c3a71 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:48.273186 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 132/246] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:48.274422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c81c-c69b-488e-8855-324b63ce0e6b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:48.315357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:48.345949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:48.403129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4c81c-c69b-488e-8855-324b63ce0e6b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:48.403354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4681ec31-06e4-4b74-a37b-8555278c3a71 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:48.444228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafc925-06e8-4502-8ff8-b2088b8f9e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:48.445000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e37f-1648-4820-85bb-95450e2464ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:48.445196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafc925-06e8-4502-8ff8-b2088b8f9e06 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:48.445749 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafc925-06e8-4502-8ff8-b2088b8f9e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:48.446094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eafc925-06e8-4502-8ff8-b2088b8f9e06 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:48.446461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e37f-1648-4820-85bb-95450e2464ca None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:48.446971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e37f-1648-4820-85bb-95450e2464ca None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:48.447426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e37f-1648-4820-85bb-95450e2464ca None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:48.447576 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafc925-06e8-4502-8ff8-b2088b8f9e06 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:48.448757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e37f-1648-4820-85bb-95450e2464ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:48.449261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eafc925-06e8-4502-8ff8-b2088b8f9e06 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:48.450693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e37f-1648-4820-85bb-95450e2464ca None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:48.451219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 115/247] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:48 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:48.452196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafc925-06e8-4502-8ff8-b2088b8f9e06 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:48.478938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 133/248] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:10:48 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:48.480241 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e37f-1648-4820-85bb-95450e2464ca None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:48.643180 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e37f-1648-4820-85bb-95450e2464ca None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:48.648103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:48.694080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee2195-7df9-4b8e-9110-2c21081f1868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:48.696512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee2195-7df9-4b8e-9110-2c21081f1868 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:48.697325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee2195-7df9-4b8e-9110-2c21081f1868 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:48.697834 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee2195-7df9-4b8e-9110-2c21081f1868 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d/users/f9c63e2687f34df5ad7c08a5002b2429/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:48.698122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eafc925-06e8-4502-8ff8-b2088b8f9e06 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:48.702127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee2195-7df9-4b8e-9110-2c21081f1868 None admin] RBAC: Authorizing `identity:create_grant(project_id=366a6c5b1cb444698289e32b2b46918d, user_id=f9c63e2687f34df5ad7c08a5002b2429, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:48.705480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee2195-7df9-4b8e-9110-2c21081f1868 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:48.727431 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 134/249] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:10:48 2022] PUT /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d/users/f9c63e2687f34df5ad7c08a5002b2429/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:10:48.744551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:48.768712 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bed4df-ddba-4921-a8b0-e73379a580a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:48.770886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bed4df-ddba-4921-a8b0-e73379a580a3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:48.771408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bed4df-ddba-4921-a8b0-e73379a580a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:48.771888 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bed4df-ddba-4921-a8b0-e73379a580a3 None admin] PATH_INFO: `/v3/users/700e62458e044820b8a954db488d235a` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:48.800389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bed4df-ddba-4921-a8b0-e73379a580a3 None admin] RBAC: Authorizing `identity:get_user(user_id=700e62458e044820b8a954db488d235a)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:48.803369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bed4df-ddba-4921-a8b0-e73379a580a3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:48.806920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 116/250] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:48 2022] GET /identity/v3/users/700e62458e044820b8a954db488d235a => generated 297 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:48.812598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee2195-7df9-4b8e-9110-2c21081f1868 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:49.025855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee2195-7df9-4b8e-9110-2c21081f1868 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:49.070015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e27e2a-303f-42db-a8ea-7f02a3ba9c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:49.070822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e27e2a-303f-42db-a8ea-7f02a3ba9c13 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:49.071114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e27e2a-303f-42db-a8ea-7f02a3ba9c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:49.071401 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e27e2a-303f-42db-a8ea-7f02a3ba9c13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:49.094705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:49.161019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e27e2a-303f-42db-a8ea-7f02a3ba9c13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:49.162712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e27e2a-303f-42db-a8ea-7f02a3ba9c13 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:49.283064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 135/251] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:49.288922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bed4df-ddba-4921-a8b0-e73379a580a3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:49.415228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bed4df-ddba-4921-a8b0-e73379a580a3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:49.440809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2af36-9c2b-4c09-858d-28db1ceed2f3 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:49.441153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2af36-9c2b-4c09-858d-28db1ceed2f3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:49.441546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2af36-9c2b-4c09-858d-28db1ceed2f3 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:49.444368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 136/252] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:10:49.450825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5c14-3ee9-4bb5-a4e7-1a48366e55bd None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:49.451135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5c14-3ee9-4bb5-a4e7-1a48366e55bd None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:49.451398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f5c14-3ee9-4bb5-a4e7-1a48366e55bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:49.468514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c8545-3ab2-4dbf-86b3-a48655f0c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:49.469455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c8545-3ab2-4dbf-86b3-a48655f0c283 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:49.469746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c8545-3ab2-4dbf-86b3-a48655f0c283 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:49.470029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55c8545-3ab2-4dbf-86b3-a48655f0c283 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:49.471318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-302f5c14-3ee9-4bb5-a4e7-1a48366e55bd None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:49.471443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c8545-3ab2-4dbf-86b3-a48655f0c283 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:49.472526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-302f5c14-3ee9-4bb5-a4e7-1a48366e55bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:49.473254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55c8545-3ab2-4dbf-86b3-a48655f0c283 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:49.475309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 117/253] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:49 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:49.481522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c8545-3ab2-4dbf-86b3-a48655f0c283 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:49.542905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:49.559220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:49.605538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 137/254] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:10:49.646803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55c8545-3ab2-4dbf-86b3-a48655f0c283 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:49.647670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f5c14-3ee9-4bb5-a4e7-1a48366e55bd None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:49.720018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5a8b9-1c6f-4fd7-8772-10b51b80dd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:49.720708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a8b9-1c6f-4fd7-8772-10b51b80dd82 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:49.720998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a8b9-1c6f-4fd7-8772-10b51b80dd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:49.721281 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5a8b9-1c6f-4fd7-8772-10b51b80dd82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:49.722142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5a8b9-1c6f-4fd7-8772-10b51b80dd82 None admin] RBAC: Adding query filter params (role.id=53b551bc12e84268991ca46bce5da0ec, scope.project.id=366a6c5b1cb444698289e32b2b46918d, user.id=700e62458e044820b8a954db488d235a) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:49.722467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5a8b9-1c6f-4fd7-8772-10b51b80dd82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:49.724099 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5a8b9-1c6f-4fd7-8772-10b51b80dd82 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:49.730623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 118/255] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:49 2022] GET /identity/v3/role_assignments?user.id=700e62458e044820b8a954db488d235a&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=53b551bc12e84268991ca46bce5da0ec => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:49.809737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046d98d-fa70-4586-9435-1160539dde1d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:49.810411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046d98d-fa70-4586-9435-1160539dde1d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:49.810957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046d98d-fa70-4586-9435-1160539dde1d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:49.813305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 119/256] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:49.821126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e2a8-9aab-4ec5-bf11-e1e5b3f5141d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:49.822067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e2a8-9aab-4ec5-bf11-e1e5b3f5141d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:49.825468 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1e2a8-9aab-4ec5-bf11-e1e5b3f5141d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:49.883285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-19f1e2a8-9aab-4ec5-bf11-e1e5b3f5141d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:49.891860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-19f1e2a8-9aab-4ec5-bf11-e1e5b3f5141d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:49.956087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f5c14-3ee9-4bb5-a4e7-1a48366e55bd None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:50.042840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecd2ba-0869-4bdd-8db5-f668c5cbe30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:50.045126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecd2ba-0869-4bdd-8db5-f668c5cbe30c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:50.045955 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecd2ba-0869-4bdd-8db5-f668c5cbe30c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:50.046424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ecd2ba-0869-4bdd-8db5-f668c5cbe30c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:50.081439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ecd2ba-0869-4bdd-8db5-f668c5cbe30c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-87ecd2ba-0869-4bdd-8db5-f668c5cbe30c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:50.113189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:50.114185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:10:50.115093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:50.117712 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 120/257] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:50.118812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 138/258] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:10:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:50.128021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1e2a8-9aab-4ec5-bf11-e1e5b3f5141d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:50.176508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecd2ba-0869-4bdd-8db5-f668c5cbe30c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:50.209188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:50.315086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:50.389131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1e2a8-9aab-4ec5-bf11-e1e5b3f5141d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:50.501756 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281c9a6-88c2-4ed1-b859-30e8d089e1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:50.502462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c9a6-88c2-4ed1-b859-30e8d089e1bb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:50.502801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c9a6-88c2-4ed1-b859-30e8d089e1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:50.503087 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c9a6-88c2-4ed1-b859-30e8d089e1bb None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:50.503895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281c9a6-88c2-4ed1-b859-30e8d089e1bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:50.504827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281c9a6-88c2-4ed1-b859-30e8d089e1bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:50.506484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281c9a6-88c2-4ed1-b859-30e8d089e1bb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:50.527082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 121/259] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:10:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:50.534977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281c9a6-88c2-4ed1-b859-30e8d089e1bb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:50.543351 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ecd2ba-0869-4bdd-8db5-f668c5cbe30c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:50.655526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96587927-2cdb-4fe3-9c05-e0a38480560c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:50.656266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96587927-2cdb-4fe3-9c05-e0a38480560c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:50.656550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96587927-2cdb-4fe3-9c05-e0a38480560c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:50.656828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96587927-2cdb-4fe3-9c05-e0a38480560c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:50.700107 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96587927-2cdb-4fe3-9c05-e0a38480560c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-96587927-2cdb-4fe3-9c05-e0a38480560c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:50.724546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:50.726303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:50.727936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 139/260] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:10:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 552 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:50.740358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96587927-2cdb-4fe3-9c05-e0a38480560c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:50.878320 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281c9a6-88c2-4ed1-b859-30e8d089e1bb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:50.882240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:50.991787 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771610ca-c78e-4353-a1a0-1d97f5160be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:50.992637 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771610ca-c78e-4353-a1a0-1d97f5160be8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:50.992928 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771610ca-c78e-4353-a1a0-1d97f5160be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:50.993209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771610ca-c78e-4353-a1a0-1d97f5160be8 None admin] PATH_INFO: `/v3/users/e8f24e7c37e54fa788135577d7ace1ad` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:50.999677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771610ca-c78e-4353-a1a0-1d97f5160be8 None admin] RBAC: Authorizing `identity:get_user(user_id=e8f24e7c37e54fa788135577d7ace1ad)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:51.001556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771610ca-c78e-4353-a1a0-1d97f5160be8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:51.003900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 122/261] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:50 2022] GET /identity/v3/users/e8f24e7c37e54fa788135577d7ace1ad => generated 283 bytes in 470 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:51.015824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771610ca-c78e-4353-a1a0-1d97f5160be8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:51.113056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:51.122258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96587927-2cdb-4fe3-9c05-e0a38480560c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:51.265926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729848f4-d68b-4947-be11-6a2bdc856777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:51.267041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729848f4-d68b-4947-be11-6a2bdc856777 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:51.267546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729848f4-d68b-4947-be11-6a2bdc856777 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:51.268038 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729848f4-d68b-4947-be11-6a2bdc856777 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:51.269602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729848f4-d68b-4947-be11-6a2bdc856777 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:51.270146 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729848f4-d68b-4947-be11-6a2bdc856777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:51.277324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729848f4-d68b-4947-be11-6a2bdc856777 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:51.302732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771610ca-c78e-4353-a1a0-1d97f5160be8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:51.311174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 140/262] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 572 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:51.339919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729848f4-d68b-4947-be11-6a2bdc856777 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:51.423669 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddae12-03a1-4a88-af83-7178efa8df02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:51.424745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddae12-03a1-4a88-af83-7178efa8df02 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:51.425238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddae12-03a1-4a88-af83-7178efa8df02 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:51.426420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddae12-03a1-4a88-af83-7178efa8df02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:51.547640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:51.656498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddae12-03a1-4a88-af83-7178efa8df02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:51.659169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddae12-03a1-4a88-af83-7178efa8df02 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:51.727604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729848f4-d68b-4947-be11-6a2bdc856777 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:51.908010 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74160e-a4cd-489d-9f37-ef2ff37808a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:51.909464 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74160e-a4cd-489d-9f37-ef2ff37808a7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:51.909978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74160e-a4cd-489d-9f37-ef2ff37808a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:51.910530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74160e-a4cd-489d-9f37-ef2ff37808a7 None admin] PATH_INFO: `/v3/users/e8f24e7c37e54fa788135577d7ace1ad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:51.915995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74160e-a4cd-489d-9f37-ef2ff37808a7 None admin] RBAC: Authorizing `identity:get_user(user_id=e8f24e7c37e54fa788135577d7ace1ad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:51.918489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74160e-a4cd-489d-9f37-ef2ff37808a7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:51.923047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 141/263] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:51 2022] GET /identity/v3/users/e8f24e7c37e54fa788135577d7ace1ad => generated 283 bytes in 583 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:51.936382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74160e-a4cd-489d-9f37-ef2ff37808a7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:52.027596 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 123/264] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1011 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:52.045415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddae12-03a1-4a88-af83-7178efa8df02 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:52.079245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:52.274282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:52.366487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74160e-a4cd-489d-9f37-ef2ff37808a7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:52.396913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddae12-03a1-4a88-af83-7178efa8df02 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:52.474648 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1505d-c8ef-411c-9a9d-42b4e774ae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:52.475874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1505d-c8ef-411c-9a9d-42b4e774ae2f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:52.476324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1505d-c8ef-411c-9a9d-42b4e774ae2f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:52.476614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1505d-c8ef-411c-9a9d-42b4e774ae2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:52.539484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e732-4147-4fc1-bd59-cc086ec811b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:52.541402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e732-4147-4fc1-bd59-cc086ec811b1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:52.541912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e732-4147-4fc1-bd59-cc086ec811b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:52.542413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334e732-4147-4fc1-bd59-cc086ec811b1 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:52.544794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e732-4147-4fc1-bd59-cc086ec811b1 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:52.548562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334e732-4147-4fc1-bd59-cc086ec811b1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:52.552794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 124/265] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:52 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 508 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:52.561423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e732-4147-4fc1-bd59-cc086ec811b1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:52.689156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1505d-c8ef-411c-9a9d-42b4e774ae2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:52.693444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1505d-c8ef-411c-9a9d-42b4e774ae2f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:52.768327 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:52.854954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 142/266] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 920 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:52.856848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ade6d-dfe9-47e6-ae20-1b3d89718cc7 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:52.857349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ade6d-dfe9-47e6-ae20-1b3d89718cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:52.857788 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ade6d-dfe9-47e6-ae20-1b3d89718cc7 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:52.860127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 143/267] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:52.871243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbbdfe-f1f9-4160-9349-d546c1294aba None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:52.871897 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbbdfe-f1f9-4160-9349-d546c1294aba None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:52.872344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbbdfe-f1f9-4160-9349-d546c1294aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:52.887562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334e732-4147-4fc1-bd59-cc086ec811b1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:52.929296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-22bbbdfe-f1f9-4160-9349-d546c1294aba None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:52.931582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-22bbbdfe-f1f9-4160-9349-d546c1294aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:52.968917 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e80d1-90b4-4683-9cb4-da6c7d20777e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:52.970428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e80d1-90b4-4683-9cb4-da6c7d20777e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:52.971197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e80d1-90b4-4683-9cb4-da6c7d20777e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:52.974957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e80d1-90b4-4683-9cb4-da6c7d20777e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:52.977025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e80d1-90b4-4683-9cb4-da6c7d20777e None admin] RBAC: Adding query filter params (role.id=1d508b4effb546039e87dd273933af16, scope.project.id=366a6c5b1cb444698289e32b2b46918d, user.id=e8f24e7c37e54fa788135577d7ace1ad) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:52.979445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e80d1-90b4-4683-9cb4-da6c7d20777e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:52.982560 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e80d1-90b4-4683-9cb4-da6c7d20777e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:53.009535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 125/268] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:52 2022] GET /identity/v3/role_assignments?user.id=e8f24e7c37e54fa788135577d7ace1ad&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=1d508b4effb546039e87dd273933af16 => generated 608 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:53.017057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e80d1-90b4-4683-9cb4-da6c7d20777e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:53.035875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:53.237801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 144/269] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:53.307953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbbdfe-f1f9-4160-9349-d546c1294aba None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:53.385029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e80d1-90b4-4683-9cb4-da6c7d20777e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:53.497389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:53.538043 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ce3c8-0605-4c60-a916-6c158287abf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:53.539671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ce3c8-0605-4c60-a916-6c158287abf5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:53.541223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ce3c8-0605-4c60-a916-6c158287abf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:53.541223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ce3c8-0605-4c60-a916-6c158287abf5 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:53.549607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ce3c8-0605-4c60-a916-6c158287abf5 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:53.550363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ce3c8-0605-4c60-a916-6c158287abf5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:53.554164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 126/270] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:53 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 538 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:53.580378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ce3c8-0605-4c60-a916-6c158287abf5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:53.593205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbbdfe-f1f9-4160-9349-d546c1294aba None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:53.669916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cae5a5-9c23-416a-a933-0e25e6677565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:53.671383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae5a5-9c23-416a-a933-0e25e6677565 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:53.671851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae5a5-9c23-416a-a933-0e25e6677565 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:53.672284 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cae5a5-9c23-416a-a933-0e25e6677565 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:53.719654 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cae5a5-9c23-416a-a933-0e25e6677565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c1cae5a5-9c23-416a-a933-0e25e6677565 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:53.735675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:53.737568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:53.739872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:53.743428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 145/271] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:53.746235 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff9d99-5204-436a-bc43-c25eeba58c13 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:53.747022 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff9d99-5204-436a-bc43-c25eeba58c13 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:53.747809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff9d99-5204-436a-bc43-c25eeba58c13 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:53.751061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 146/272] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:53.754372 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e45f-f0fc-4822-a98f-95dc6972c74d None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:53.755032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e45f-f0fc-4822-a98f-95dc6972c74d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:53.755589 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8e45f-f0fc-4822-a98f-95dc6972c74d None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:53.757304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 147/273] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:53.759123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8e45f-f0fc-4822-a98f-95dc6972c74d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:53.822197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:53.883091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ce3c8-0605-4c60-a916-6c158287abf5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:53.954833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4c4d4-9f9f-4181-b4e7-5854a455a0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:53.955680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c4d4-9f9f-4181-b4e7-5854a455a0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:53.956074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c4d4-9f9f-4181-b4e7-5854a455a0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:53.956472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4c4d4-9f9f-4181-b4e7-5854a455a0e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:53.958054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4c4d4-9f9f-4181-b4e7-5854a455a0e3 None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=366a6c5b1cb444698289e32b2b46918d, user.id=e8f24e7c37e54fa788135577d7ace1ad) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:53.958480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4c4d4-9f9f-4181-b4e7-5854a455a0e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:53.960224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4c4d4-9f9f-4181-b4e7-5854a455a0e3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:53.967760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8e45f-f0fc-4822-a98f-95dc6972c74d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:53.970099 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 127/274] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:53 2022] GET /identity/v3/role_assignments?user.id=e8f24e7c37e54fa788135577d7ace1ad&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 608 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:53.975593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852460ac-2f3d-4330-870a-889dcf7c5b08 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:53.976109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852460ac-2f3d-4330-870a-889dcf7c5b08 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:53.976624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852460ac-2f3d-4330-870a-889dcf7c5b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:54.019233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:54.024394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-852460ac-2f3d-4330-870a-889dcf7c5b08 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:54.028426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-852460ac-2f3d-4330-870a-889dcf7c5b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:54.053279 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bad692-1b5b-47cd-b025-8ef67f61aa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:54.054835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bad692-1b5b-47cd-b025-8ef67f61aa19 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:54.055276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bad692-1b5b-47cd-b025-8ef67f61aa19 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:54.055703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bad692-1b5b-47cd-b025-8ef67f61aa19 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:54.056964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bad692-1b5b-47cd-b025-8ef67f61aa19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:54.057438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bad692-1b5b-47cd-b025-8ef67f61aa19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:54.059995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bad692-1b5b-47cd-b025-8ef67f61aa19 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:54.076639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 148/275] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:54.078851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d726fd1-b01c-4d8a-8434-c348a19b0db3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:54.079287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d726fd1-b01c-4d8a-8434-c348a19b0db3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:54.079714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d726fd1-b01c-4d8a-8434-c348a19b0db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:54.108839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7d726fd1-b01c-4d8a-8434-c348a19b0db3 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:54.111492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7d726fd1-b01c-4d8a-8434-c348a19b0db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:54.173793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 128/276] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:54.177653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86735cb7-67d6-4ae7-9c1c-3917095eecbe None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:54.178158 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86735cb7-67d6-4ae7-9c1c-3917095eecbe None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:54.178717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86735cb7-67d6-4ae7-9c1c-3917095eecbe None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:54.181440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 129/277] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:54.189982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86735cb7-67d6-4ae7-9c1c-3917095eecbe None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:54.257456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 149/278] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:54.259975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7454d-1022-4a9a-9112-973725f4a247 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:54.260482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7454d-1022-4a9a-9112-973725f4a247 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:54.260983 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7454d-1022-4a9a-9112-973725f4a247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:54.291393 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c9f7454d-1022-4a9a-9112-973725f4a247 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:54.294738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c9f7454d-1022-4a9a-9112-973725f4a247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:54.337074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:54.399307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86735cb7-67d6-4ae7-9c1c-3917095eecbe None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:54.426607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 150/279] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:54.428149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7454d-1022-4a9a-9112-973725f4a247 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:54.457316 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98501090-0127-4215-945e-806f0ba401de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:54.458960 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98501090-0127-4215-945e-806f0ba401de None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:54.459440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98501090-0127-4215-945e-806f0ba401de None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:54.459994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98501090-0127-4215-945e-806f0ba401de None admin] PATH_INFO: `/v3/users/f9c63e2687f34df5ad7c08a5002b2429` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:54.462463 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98501090-0127-4215-945e-806f0ba401de None admin] RBAC: Authorizing `identity:get_user(user_id=f9c63e2687f34df5ad7c08a5002b2429)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:54.465423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98501090-0127-4215-945e-806f0ba401de None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:54.468528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 130/280] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:54 2022] GET /identity/v3/users/f9c63e2687f34df5ad7c08a5002b2429 => generated 297 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:54.469226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98501090-0127-4215-945e-806f0ba401de None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:54.499068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:54.597283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7454d-1022-4a9a-9112-973725f4a247 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:54.665938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d93e41-f567-4835-a282-678bee5bf3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:54.667509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d93e41-f567-4835-a282-678bee5bf3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:54.667961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d93e41-f567-4835-a282-678bee5bf3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:54.668469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d93e41-f567-4835-a282-678bee5bf3e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:54.677908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98501090-0127-4215-945e-806f0ba401de None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:54.694668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d93e41-f567-4835-a282-678bee5bf3e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-93d93e41-f567-4835-a282-678bee5bf3e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:54.710487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:54.712291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 151/281] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:54.726225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d93e41-f567-4835-a282-678bee5bf3e0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:54.779701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:54.790860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a5dc5-98f5-4a6e-94b7-bf41a519120e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:54.792255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a5dc5-98f5-4a6e-94b7-bf41a519120e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:54.792762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a5dc5-98f5-4a6e-94b7-bf41a519120e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:54.793258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a5dc5-98f5-4a6e-94b7-bf41a519120e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:54.825029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a5dc5-98f5-4a6e-94b7-bf41a519120e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9f0a5dc5-98f5-4a6e-94b7-bf41a519120e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:54.840859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:54.842922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:10:54.844534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 131/282] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:10:54.846096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a5dc5-98f5-4a6e-94b7-bf41a519120e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:54.926216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d93e41-f567-4835-a282-678bee5bf3e0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:54.929268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:54.995666 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512a926-6c15-4b48-b99d-74b48f591cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:54.997176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512a926-6c15-4b48-b99d-74b48f591cac None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:54.997673 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512a926-6c15-4b48-b99d-74b48f591cac None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:54.998169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512a926-6c15-4b48-b99d-74b48f591cac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:55.000476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512a926-6c15-4b48-b99d-74b48f591cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:55.004534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512a926-6c15-4b48-b99d-74b48f591cac None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:55.007748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 152/283] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:55.009244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512a926-6c15-4b48-b99d-74b48f591cac None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:55.030904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a5dc5-98f5-4a6e-94b7-bf41a519120e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:55.094476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58c1b0-6fc9-4960-be9b-228d9f34fd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:55.095643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58c1b0-6fc9-4960-be9b-228d9f34fd5a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:55.096167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58c1b0-6fc9-4960-be9b-228d9f34fd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:55.096666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58c1b0-6fc9-4960-be9b-228d9f34fd5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:55.179710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512a926-6c15-4b48-b99d-74b48f591cac None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:55.223953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20fca2-d311-46f3-ac31-eadd196b9120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:55.224953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20fca2-d311-46f3-ac31-eadd196b9120 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:55.225246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20fca2-d311-46f3-ac31-eadd196b9120 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:55.225546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20fca2-d311-46f3-ac31-eadd196b9120 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:55.226647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20fca2-d311-46f3-ac31-eadd196b9120 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:55.227061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20fca2-d311-46f3-ac31-eadd196b9120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:55.229618 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:55.229925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20fca2-d311-46f3-ac31-eadd196b9120 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:55.245708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 153/284] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:55.247296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20fca2-d311-46f3-ac31-eadd196b9120 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:55.251093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58c1b0-6fc9-4960-be9b-228d9f34fd5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:55.254564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58c1b0-6fc9-4960-be9b-228d9f34fd5a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:55.318453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:55.446999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20fca2-d311-46f3-ac31-eadd196b9120 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:55.461445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 132/285] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:10:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:10:55.462460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58c1b0-6fc9-4960-be9b-228d9f34fd5a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:55.513296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60f882-a13e-43b5-95d8-2cdc6c925ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:55.514488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f882-a13e-43b5-95d8-2cdc6c925ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:55.515053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f882-a13e-43b5-95d8-2cdc6c925ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:55.515559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60f882-a13e-43b5-95d8-2cdc6c925ba4 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:55.517080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60f882-a13e-43b5-95d8-2cdc6c925ba4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:55.517657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60f882-a13e-43b5-95d8-2cdc6c925ba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:55.521054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60f882-a13e-43b5-95d8-2cdc6c925ba4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:55.538368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 154/286] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:55.539404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60f882-a13e-43b5-95d8-2cdc6c925ba4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:55.653668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:55.673540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58c1b0-6fc9-4960-be9b-228d9f34fd5a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:55.718450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:55.733430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60f882-a13e-43b5-95d8-2cdc6c925ba4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:55.743878 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf66f3-f9df-4211-8592-0102a142934e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:55.745080 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf66f3-f9df-4211-8592-0102a142934e None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:55.745598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf66f3-f9df-4211-8592-0102a142934e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:55.746098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf66f3-f9df-4211-8592-0102a142934e None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:55.747474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf66f3-f9df-4211-8592-0102a142934e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:55.749249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf66f3-f9df-4211-8592-0102a142934e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:55.781488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 133/287] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:10:55 2022] POST /identity/v3/users => generated 293 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:55.783238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf66f3-f9df-4211-8592-0102a142934e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:55.793835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d53e43-1c8e-4ac5-9d15-779bdceeebdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:55.795465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d53e43-1c8e-4ac5-9d15-779bdceeebdf None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:55.795994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d53e43-1c8e-4ac5-9d15-779bdceeebdf None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:55.796390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d53e43-1c8e-4ac5-9d15-779bdceeebdf None admin] PATH_INFO: `/v3/users/700e62458e044820b8a954db488d235a` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:55.798880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d53e43-1c8e-4ac5-9d15-779bdceeebdf None admin] RBAC: Authorizing `identity:get_user(user_id=700e62458e044820b8a954db488d235a)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:55.801680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d53e43-1c8e-4ac5-9d15-779bdceeebdf None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:55.805173 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 155/288] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:55 2022] GET /identity/v3/users/700e62458e044820b8a954db488d235a => generated 297 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:55.806364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d53e43-1c8e-4ac5-9d15-779bdceeebdf None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:55.996972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf66f3-f9df-4211-8592-0102a142934e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:56.017883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d53e43-1c8e-4ac5-9d15-779bdceeebdf None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:56.077048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6e679-0eed-4d38-ae26-468f4ef5e5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:56.078331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e679-0eed-4d38-ae26-468f4ef5e5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:56.078767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e679-0eed-4d38-ae26-468f4ef5e5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:56.084424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6e679-0eed-4d38-ae26-468f4ef5e5f7 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:56.084424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6e679-0eed-4d38-ae26-468f4ef5e5f7 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:56.084754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6e679-0eed-4d38-ae26-468f4ef5e5f7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:56.086782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 134/289] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:55 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:56.087776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6e679-0eed-4d38-ae26-468f4ef5e5f7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:56.111305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5e44f-9f45-456b-a924-c248a661706c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:56.112860 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5e44f-9f45-456b-a924-c248a661706c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:56.114040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5e44f-9f45-456b-a924-c248a661706c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:56.114888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5e44f-9f45-456b-a924-c248a661706c None admin] PATH_INFO: `/v3/users/b493ca31dac24b3289024bd29e53e249` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:56.132782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:56.146621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5e44f-9f45-456b-a924-c248a661706c None admin] RBAC: Authorizing `identity:get_user(user_id=b493ca31dac24b3289024bd29e53e249)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:56.149302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5e44f-9f45-456b-a924-c248a661706c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:56.152852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 156/290] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:10:55 2022] GET /identity/v3/users/b493ca31dac24b3289024bd29e53e249 => generated 293 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:56.154188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5e44f-9f45-456b-a924-c248a661706c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:56.202272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:56.314038 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6e679-0eed-4d38-ae26-468f4ef5e5f7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:56.352770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5e44f-9f45-456b-a924-c248a661706c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:56.391669 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f84a41-2c4e-41b1-9af8-d7b7d395f870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:56.393088 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f84a41-2c4e-41b1-9af8-d7b7d395f870 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:56.393553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f84a41-2c4e-41b1-9af8-d7b7d395f870 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:56.394019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f84a41-2c4e-41b1-9af8-d7b7d395f870 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:56.396431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f84a41-2c4e-41b1-9af8-d7b7d395f870 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:56.398909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f84a41-2c4e-41b1-9af8-d7b7d395f870 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:56.401681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 135/291] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:10:56 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:56.402880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f84a41-2c4e-41b1-9af8-d7b7d395f870 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:56.433830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb59901-7ada-459f-b207-847b87add0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:56.435136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59901-7ada-459f-b207-847b87add0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:56.435605 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59901-7ada-459f-b207-847b87add0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:56.436084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb59901-7ada-459f-b207-847b87add0c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:56.437876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb59901-7ada-459f-b207-847b87add0c9 None admin] RBAC: Adding query filter params (role.id=ef48ab9c5b844be59351a260130d3071, scope.project.id=366a6c5b1cb444698289e32b2b46918d, user.id=f9c63e2687f34df5ad7c08a5002b2429) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:56.438391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb59901-7ada-459f-b207-847b87add0c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:56.441884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb59901-7ada-459f-b207-847b87add0c9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:56.454405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 157/292] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:10:56 2022] GET /identity/v3/role_assignments?user.id=f9c63e2687f34df5ad7c08a5002b2429&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=ef48ab9c5b844be59351a260130d3071 => generated 608 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:56.456077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb59901-7ada-459f-b207-847b87add0c9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:56.578331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:56.610741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f84a41-2c4e-41b1-9af8-d7b7d395f870 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:56.633068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:56.671379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb59901-7ada-459f-b207-847b87add0c9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:56.701205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a109c-8f08-4b8e-bf44-1130c1dd37c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:56.702821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a109c-8f08-4b8e-bf44-1130c1dd37c0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:56.703563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a109c-8f08-4b8e-bf44-1130c1dd37c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:56.704248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a109c-8f08-4b8e-bf44-1130c1dd37c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:56.705884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a109c-8f08-4b8e-bf44-1130c1dd37c0 None admin] RBAC: Adding query filter params (role.id=ef48ab9c5b844be59351a260130d3071, user.id=b493ca31dac24b3289024bd29e53e249, scope.system=all) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:56.706607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a109c-8f08-4b8e-bf44-1130c1dd37c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:56.709177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a109c-8f08-4b8e-bf44-1130c1dd37c0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:56.719762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 136/293] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Jul 20 08:10:56 2022] GET /identity/v3/role_assignments?user.id=b493ca31dac24b3289024bd29e53e249&scope.system=all&role.id=ef48ab9c5b844be59351a260130d3071 => generated 226 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:56.775647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81f832-0f55-4cb4-ad67-68af24d35db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:56.776800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81f832-0f55-4cb4-ad67-68af24d35db1 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:56.777095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81f832-0f55-4cb4-ad67-68af24d35db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:56.777378 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81f832-0f55-4cb4-ad67-68af24d35db1 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d/users/700e62458e044820b8a954db488d235a/roles/53b551bc12e84268991ca46bce5da0ec` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:56.779842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81f832-0f55-4cb4-ad67-68af24d35db1 None admin] RBAC: Authorizing `identity:create_grant(project_id=366a6c5b1cb444698289e32b2b46918d, user_id=700e62458e044820b8a954db488d235a, role_id=53b551bc12e84268991ca46bce5da0ec)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:56.781646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81f832-0f55-4cb4-ad67-68af24d35db1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:56.821216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 158/294] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:10:56 2022] PUT /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d/users/700e62458e044820b8a954db488d235a/roles/53b551bc12e84268991ca46bce5da0ec => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:10:58.865733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbbb13-ce4d-4e25-8081-3d36fe66e41d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:58.866428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbbb13-ce4d-4e25-8081-3d36fe66e41d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:58.866428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbbb13-ce4d-4e25-8081-3d36fe66e41d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:58.868407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 137/295] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:58.874625 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d5612-ebcb-4384-a873-89ed94a51ecb None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:58.875247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d5612-ebcb-4384-a873-89ed94a51ecb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:58.875247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d5612-ebcb-4384-a873-89ed94a51ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:58.895889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c49d5612-ebcb-4384-a873-89ed94a51ecb None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:58.897116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c49d5612-ebcb-4384-a873-89ed94a51ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:59.018213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 159/296] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:59.048045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbbb13-ce4d-4e25-8081-3d36fe66e41d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:59.078781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c51914-b690-46b5-9d99-21cbefd6322c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:59.079478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c51914-b690-46b5-9d99-21cbefd6322c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:59.080268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c51914-b690-46b5-9d99-21cbefd6322c None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:59.084485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 160/297] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:59.092792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0a0f0-43bd-4da1-ae7f-2a5e468c21a2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:59.093699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0a0f0-43bd-4da1-ae7f-2a5e468c21a2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:59.094452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0a0f0-43bd-4da1-ae7f-2a5e468c21a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:59.123727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-99e0a0f0-43bd-4da1-ae7f-2a5e468c21a2 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:59.126216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-99e0a0f0-43bd-4da1-ae7f-2a5e468c21a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:59.240417 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 161/298] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:10:59.247681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbbb13-ce4d-4e25-8081-3d36fe66e41d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:59.266355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0a0f0-43bd-4da1-ae7f-2a5e468c21a2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:59.273772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:59.310572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216b75e-f40a-4140-ad66-b75b7134cbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:59.311411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b75e-f40a-4140-ad66-b75b7134cbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:59.312036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b75e-f40a-4140-ad66-b75b7134cbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:59.312470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b75e-f40a-4140-ad66-b75b7134cbd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:59.314236 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216b75e-f40a-4140-ad66-b75b7134cbd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:59.316688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216b75e-f40a-4140-ad66-b75b7134cbd5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:59.319030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 138/299] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:10:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:10:59.334619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216b75e-f40a-4140-ad66-b75b7134cbd5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:59.351898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:59.507513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0a0f0-43bd-4da1-ae7f-2a5e468c21a2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:59.516235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216b75e-f40a-4140-ad66-b75b7134cbd5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:59.584983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fe66b-dbdd-477a-8925-2c960a63cd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:59.586196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fe66b-dbdd-477a-8925-2c960a63cd27 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:59.586948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fe66b-dbdd-477a-8925-2c960a63cd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:59.587681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fe66b-dbdd-477a-8925-2c960a63cd27 None admin] PATH_INFO: `/v3/groups` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:59.588927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fe66b-dbdd-477a-8925-2c960a63cd27 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:59.591103 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926fe66b-dbdd-477a-8925-2c960a63cd27 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:59.597712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c851d6-a45e-40ba-91fc-89a7a985ef2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:59.599192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c851d6-a45e-40ba-91fc-89a7a985ef2a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:59.599675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c851d6-a45e-40ba-91fc-89a7a985ef2a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:59.600161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c851d6-a45e-40ba-91fc-89a7a985ef2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:59.621623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c851d6-a45e-40ba-91fc-89a7a985ef2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:10:59.627215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 139/300] 10.222.0.64 () {60 vars in 1134 bytes} [Wed Jul 20 08:10:59 2022] POST /identity/v3/groups => generated 229 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c2c851d6-a45e-40ba-91fc-89a7a985ef2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:10:59.634028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:10:59.636070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:10:59.637307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:10:59.640889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 162/301] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:10:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:10:59.648700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fe66b-dbdd-477a-8925-2c960a63cd27 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:10:59.766748 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:10:59.810154 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c09040-7eb9-435d-9dbd-32ff72569a01 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:59.810481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c09040-7eb9-435d-9dbd-32ff72569a01 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:59.810705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c09040-7eb9-435d-9dbd-32ff72569a01 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:59.813466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926fe66b-dbdd-477a-8925-2c960a63cd27 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:10:59.817223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 163/302] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:59.819133 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dea5e-bef7-4d5b-bb24-b7a46cbd2f18 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:59.819612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dea5e-bef7-4d5b-bb24-b7a46cbd2f18 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:59.820077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550dea5e-bef7-4d5b-bb24-b7a46cbd2f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:59.853918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-550dea5e-bef7-4d5b-bb24-b7a46cbd2f18 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:59.857925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-550dea5e-bef7-4d5b-bb24-b7a46cbd2f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:10:59.885104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255a78a-5b69-4930-9771-05e60f4df138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:10:59.885829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a78a-5b69-4930-9771-05e60f4df138 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:59.886087 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a78a-5b69-4930-9771-05e60f4df138 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:59.886387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255a78a-5b69-4930-9771-05e60f4df138 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:59.888487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255a78a-5b69-4930-9771-05e60f4df138 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:10:59.888808 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255a78a-5b69-4930-9771-05e60f4df138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:10:59.890297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6255a78a-5b69-4930-9771-05e60f4df138 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:10:59.904932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 140/303] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:10:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:10:59.906365 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9618094-3698-4a4e-9c94-73d63a14154e None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:59.906748 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9618094-3698-4a4e-9c94-73d63a14154e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:59.907046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9618094-3698-4a4e-9c94-73d63a14154e None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:59.908769 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 141/304] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:10:59.915702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d0c83-1469-497e-bc68-7303da3c1b1d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:10:59.916428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d0c83-1469-497e-bc68-7303da3c1b1d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:10:59.916428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d0c83-1469-497e-bc68-7303da3c1b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:10:59.936319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-438d0c83-1469-497e-bc68-7303da3c1b1d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:10:59.937953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-438d0c83-1469-497e-bc68-7303da3c1b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:00.002008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:00.018079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 164/305] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:11:00.022859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dea5e-bef7-4d5b-bb24-b7a46cbd2f18 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:00.075555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 142/306] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:11:00.075555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d0c83-1469-497e-bc68-7303da3c1b1d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:00.195314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550dea5e-bef7-4d5b-bb24-b7a46cbd2f18 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:00.208748 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:00.245281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada104a6-7b4b-471c-897d-8ba45dc5027a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:00.246491 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada104a6-7b4b-471c-897d-8ba45dc5027a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:00.246850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada104a6-7b4b-471c-897d-8ba45dc5027a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:00.247165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada104a6-7b4b-471c-897d-8ba45dc5027a None admin] PATH_INFO: `/v3/users/3fd577b795dc4a5e851c7e64887803e0` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:00.261275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438d0c83-1469-497e-bc68-7303da3c1b1d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:00.274672 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada104a6-7b4b-471c-897d-8ba45dc5027a None admin] RBAC: Authorizing `identity:get_user(user_id=3fd577b795dc4a5e851c7e64887803e0)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:00.277274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada104a6-7b4b-471c-897d-8ba45dc5027a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:00.286380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 165/307] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:11:00 2022] GET /identity/v3/users/3fd577b795dc4a5e851c7e64887803e0 => generated 293 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:00.286380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada104a6-7b4b-471c-897d-8ba45dc5027a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:00.318649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b35a58-3d34-418b-8bf7-f5813fd06af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:00.320205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b35a58-3d34-418b-8bf7-f5813fd06af2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:00.320824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b35a58-3d34-418b-8bf7-f5813fd06af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:00.321222 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b35a58-3d34-418b-8bf7-f5813fd06af2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:00.340035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b35a58-3d34-418b-8bf7-f5813fd06af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b6b35a58-3d34-418b-8bf7-f5813fd06af2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:00.353983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:00.355275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:00.356173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:00.356868 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 143/308] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:00.363246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b35a58-3d34-418b-8bf7-f5813fd06af2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:00.392806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:00.483987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada104a6-7b4b-471c-897d-8ba45dc5027a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:00.539498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801e8f-ab04-4460-b1fe-ab579799f6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:00.540392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801e8f-ab04-4460-b1fe-ab579799f6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:00.540865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801e8f-ab04-4460-b1fe-ab579799f6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:00.541323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac801e8f-ab04-4460-b1fe-ab579799f6d7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:00.547292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b35a58-3d34-418b-8bf7-f5813fd06af2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:00.556086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac801e8f-ab04-4460-b1fe-ab579799f6d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ac801e8f-ab04-4460-b1fe-ab579799f6d7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:00.564211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:00.565207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:00.566138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 166/309] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:00.567862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801e8f-ab04-4460-b1fe-ab579799f6d7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:00.597094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:00.612535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b12cc4-7def-426a-bb05-1eb3db07603d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:00.613648 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b12cc4-7def-426a-bb05-1eb3db07603d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:00.614113 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b12cc4-7def-426a-bb05-1eb3db07603d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:00.614566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b12cc4-7def-426a-bb05-1eb3db07603d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:00.616144 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b12cc4-7def-426a-bb05-1eb3db07603d None admin] RBAC: Adding query filter params (role.id=53b551bc12e84268991ca46bce5da0ec, user.id=3fd577b795dc4a5e851c7e64887803e0, scope.system=all) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:00.616665 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b12cc4-7def-426a-bb05-1eb3db07603d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:00.619479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b12cc4-7def-426a-bb05-1eb3db07603d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:00.635226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 144/310] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Jul 20 08:11:00 2022] GET /identity/v3/role_assignments?user.id=3fd577b795dc4a5e851c7e64887803e0&scope.system=all&role.id=53b551bc12e84268991ca46bce5da0ec => generated 226 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:00.637175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b12cc4-7def-426a-bb05-1eb3db07603d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:00.782207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac801e8f-ab04-4460-b1fe-ab579799f6d7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:00.812508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:00.849527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b12cc4-7def-426a-bb05-1eb3db07603d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:00.873794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d933375-d7a6-4d12-8d2f-99d4a9f359a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:00.874549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d933375-d7a6-4d12-8d2f-99d4a9f359a4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:00.874884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d933375-d7a6-4d12-8d2f-99d4a9f359a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:00.875166 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d933375-d7a6-4d12-8d2f-99d4a9f359a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:00.875933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d933375-d7a6-4d12-8d2f-99d4a9f359a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:00.876666 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d933375-d7a6-4d12-8d2f-99d4a9f359a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:00.878005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d933375-d7a6-4d12-8d2f-99d4a9f359a4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:00.888667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 167/311] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:11:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:00.894189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6bb94-3996-4d3b-9c87-4abbbe90b775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:00.896208 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6bb94-3996-4d3b-9c87-4abbbe90b775 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:00.896737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6bb94-3996-4d3b-9c87-4abbbe90b775 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:00.897250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc6bb94-3996-4d3b-9c87-4abbbe90b775 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:00.898205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6bb94-3996-4d3b-9c87-4abbbe90b775 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:00.898707 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6bb94-3996-4d3b-9c87-4abbbe90b775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:00.899417 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d933375-d7a6-4d12-8d2f-99d4a9f359a4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:00.900454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc6bb94-3996-4d3b-9c87-4abbbe90b775 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:00.919149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 145/312] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:11:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:00.927269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6bb94-3996-4d3b-9c87-4abbbe90b775 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:01.015741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:01.104419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d933375-d7a6-4d12-8d2f-99d4a9f359a4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:01.115841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc6bb94-3996-4d3b-9c87-4abbbe90b775 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:01.170777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a29caf-de28-4007-b949-a5f7cce975b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:01.172480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a29caf-de28-4007-b949-a5f7cce975b5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:01.172985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a29caf-de28-4007-b949-a5f7cce975b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:01.173336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb4811-b472-464a-ba38-711c22f8d9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:01.173545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a29caf-de28-4007-b949-a5f7cce975b5 None admin] PATH_INFO: `/v3/users/b493ca31dac24b3289024bd29e53e249` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:01.175129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb4811-b472-464a-ba38-711c22f8d9ce None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:01.176206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a29caf-de28-4007-b949-a5f7cce975b5 None admin] RBAC: Authorizing `identity:get_user(user_id=b493ca31dac24b3289024bd29e53e249)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:01.176367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb4811-b472-464a-ba38-711c22f8d9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:01.176640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb4811-b472-464a-ba38-711c22f8d9ce None admin] PATH_INFO: `/v3/users/700e62458e044820b8a954db488d235a` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:01.178959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb4811-b472-464a-ba38-711c22f8d9ce None admin] RBAC: Authorizing `identity:get_user(user_id=700e62458e044820b8a954db488d235a)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:01.179240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a29caf-de28-4007-b949-a5f7cce975b5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:01.181938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb4811-b472-464a-ba38-711c22f8d9ce None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:01.183001 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 168/313] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:11:00 2022] GET /identity/v3/users/b493ca31dac24b3289024bd29e53e249 => generated 293 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:01.184985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 146/314] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:11:00 2022] GET /identity/v3/users/700e62458e044820b8a954db488d235a => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:01.192315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a29caf-de28-4007-b949-a5f7cce975b5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:01.192970 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb4811-b472-464a-ba38-711c22f8d9ce None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:01.249525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:01.327663 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a29caf-de28-4007-b949-a5f7cce975b5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:01.372766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb0760-06d7-419f-856b-f216382ffafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:01.373818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb0760-06d7-419f-856b-f216382ffafd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:01.374482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb0760-06d7-419f-856b-f216382ffafd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:01.375067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb0760-06d7-419f-856b-f216382ffafd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:01.399998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb4811-b472-464a-ba38-711c22f8d9ce None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:01.446630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:01.464810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437628d-f9aa-4f4e-a399-e67992bd38c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:01.466906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437628d-f9aa-4f4e-a399-e67992bd38c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:01.467572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437628d-f9aa-4f4e-a399-e67992bd38c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:01.468101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437628d-f9aa-4f4e-a399-e67992bd38c5 None admin] PATH_INFO: `/v3/system/users/b493ca31dac24b3289024bd29e53e249/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:01.470647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437628d-f9aa-4f4e-a399-e67992bd38c5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b493ca31dac24b3289024bd29e53e249, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:01.474375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437628d-f9aa-4f4e-a399-e67992bd38c5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:01.489733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 147/315] 10.222.0.64 () {58 vars in 1334 bytes} [Wed Jul 20 08:11:01 2022] PUT /identity/v3/system/users/b493ca31dac24b3289024bd29e53e249/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:11:01.503151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb0760-06d7-419f-856b-f216382ffafd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:01.507155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb0760-06d7-419f-856b-f216382ffafd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:01.607320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:01.741683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 169/316] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:01.747534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437628d-f9aa-4f4e-a399-e67992bd38c5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:01.926351 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437628d-f9aa-4f4e-a399-e67992bd38c5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:01.976881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357c8789-add3-47bc-8ab7-cd1e007072a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:01.978495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c8789-add3-47bc-8ab7-cd1e007072a0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:01.979098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c8789-add3-47bc-8ab7-cd1e007072a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:01.979834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357c8789-add3-47bc-8ab7-cd1e007072a0 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:01.982442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c8789-add3-47bc-8ab7-cd1e007072a0 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:01.984549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357c8789-add3-47bc-8ab7-cd1e007072a0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:01.987548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 148/317] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:01 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:01.994314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb0760-06d7-419f-856b-f216382ffafd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:02.227444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb0760-06d7-419f-856b-f216382ffafd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:02.306330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a8248-d5b9-4d3c-b86d-f5302d54c88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:02.310620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a8248-d5b9-4d3c-b86d-f5302d54c88c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:02.310620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a8248-d5b9-4d3c-b86d-f5302d54c88c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:02.310620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a8248-d5b9-4d3c-b86d-f5302d54c88c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:02.310620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a8248-d5b9-4d3c-b86d-f5302d54c88c None admin] RBAC: Adding query filter params (role.id=53b551bc12e84268991ca46bce5da0ec, scope.project.id=366a6c5b1cb444698289e32b2b46918d, user.id=700e62458e044820b8a954db488d235a) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:02.314615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a8248-d5b9-4d3c-b86d-f5302d54c88c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:02.316958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a8248-d5b9-4d3c-b86d-f5302d54c88c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:02.326613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 170/318] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:11:01 2022] GET /identity/v3/role_assignments?user.id=700e62458e044820b8a954db488d235a&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=53b551bc12e84268991ca46bce5da0ec => generated 608 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:02.468188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064d5e-ceb9-4b88-9067-df17448d99fe None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:02.468188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064d5e-ceb9-4b88-9067-df17448d99fe None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:02.468188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8064d5e-ceb9-4b88-9067-df17448d99fe None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:02.469109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 149/319] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:11:02.477445 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8c16e-1646-42e9-99cf-72269cffe6e6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:02.477951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8c16e-1646-42e9-99cf-72269cffe6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:02.478752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8c16e-1646-42e9-99cf-72269cffe6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:02.515835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:02.536610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-02a8c16e-1646-42e9-99cf-72269cffe6e6 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:02.539280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-02a8c16e-1646-42e9-99cf-72269cffe6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:02.724462 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 171/320] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:02.750260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8064d5e-ceb9-4b88-9067-df17448d99fe None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:02.870668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:02.973700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8064d5e-ceb9-4b88-9067-df17448d99fe None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:03.048356 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c67e5-54a3-465e-9f5d-a7bc8372964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:03.049528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c67e5-54a3-465e-9f5d-a7bc8372964d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:03.050053 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c67e5-54a3-465e-9f5d-a7bc8372964d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:03.050744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c67e5-54a3-465e-9f5d-a7bc8372964d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:03.052987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c67e5-54a3-465e-9f5d-a7bc8372964d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:03.055801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c67e5-54a3-465e-9f5d-a7bc8372964d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:03.064775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 150/321] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:03.067765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8c16e-1646-42e9-99cf-72269cffe6e6 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:03.268496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8c16e-1646-42e9-99cf-72269cffe6e6 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:03.341213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11403b4-543d-445f-a699-946fd8601388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:03.342632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11403b4-543d-445f-a699-946fd8601388 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:03.342907 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11403b4-543d-445f-a699-946fd8601388 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:03.343404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11403b4-543d-445f-a699-946fd8601388 None admin] PATH_INFO: `/v3/groups` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:03.344740 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11403b4-543d-445f-a699-946fd8601388 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:03.347718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11403b4-543d-445f-a699-946fd8601388 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:03.356731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:03.388971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 172/322] 10.222.0.64 () {60 vars in 1134 bytes} [Wed Jul 20 08:11:03 2022] POST /identity/v3/groups => generated 226 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:11:03.615093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01c68e-ab97-4233-97ad-2b8f5de23f0c None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:03.615325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01c68e-ab97-4233-97ad-2b8f5de23f0c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:03.615571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01c68e-ab97-4233-97ad-2b8f5de23f0c None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:03.617091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 151/323] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:03.624487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08de3e-9b77-44f7-87fd-1b7e0355f0e0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:03.624910 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08de3e-9b77-44f7-87fd-1b7e0355f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:03.625061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08de3e-9b77-44f7-87fd-1b7e0355f0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:03.646466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-fc08de3e-9b77-44f7-87fd-1b7e0355f0e0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:03.647782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fc08de3e-9b77-44f7-87fd-1b7e0355f0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:03.791939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 173/324] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:03.832456 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01c68e-ab97-4233-97ad-2b8f5de23f0c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:04.042101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01c68e-ab97-4233-97ad-2b8f5de23f0c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:04.090312 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:04.112415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5f6e3-4263-4125-9e32-1a76f7ff5fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:04.113992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5f6e3-4263-4125-9e32-1a76f7ff5fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:04.114506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5f6e3-4263-4125-9e32-1a76f7ff5fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:04.115112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5f6e3-4263-4125-9e32-1a76f7ff5fb3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:04.141325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5f6e3-4263-4125-9e32-1a76f7ff5fb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-55f5f6e3-4263-4125-9e32-1a76f7ff5fb3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:04.158232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:04.159212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:11:04.160428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:04.161641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 152/325] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:04.170416 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08de3e-9b77-44f7-87fd-1b7e0355f0e0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:04.394029 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc08de3e-9b77-44f7-87fd-1b7e0355f0e0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:04.451817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:04.470283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355096e0-d826-4adc-b499-73c5f8b2a2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:04.471523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355096e0-d826-4adc-b499-73c5f8b2a2ac None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:04.472092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355096e0-d826-4adc-b499-73c5f8b2a2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:04.472601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355096e0-d826-4adc-b499-73c5f8b2a2ac None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:04.474143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355096e0-d826-4adc-b499-73c5f8b2a2ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:04.474698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355096e0-d826-4adc-b499-73c5f8b2a2ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:04.477040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355096e0-d826-4adc-b499-73c5f8b2a2ac None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:04.500944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 174/326] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:11:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:04.508734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5f6e3-4263-4125-9e32-1a76f7ff5fb3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:04.704314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5f6e3-4263-4125-9e32-1a76f7ff5fb3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:04.780050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5da1deb-3882-495d-ba00-75c2304dadcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:04.781680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da1deb-3882-495d-ba00-75c2304dadcd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:04.782217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da1deb-3882-495d-ba00-75c2304dadcd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:04.782768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5da1deb-3882-495d-ba00-75c2304dadcd None admin] PATH_INFO: `/v3/users/3fd577b795dc4a5e851c7e64887803e0` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:04.788174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da1deb-3882-495d-ba00-75c2304dadcd None admin] RBAC: Authorizing `identity:get_user(user_id=3fd577b795dc4a5e851c7e64887803e0)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:04.790982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5da1deb-3882-495d-ba00-75c2304dadcd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:04.794567 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 153/327] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:11:04 2022] GET /identity/v3/users/3fd577b795dc4a5e851c7e64887803e0 => generated 293 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:04.804173 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355096e0-d826-4adc-b499-73c5f8b2a2ac None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:04.806934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7fc2e-2789-4ec4-8c0d-25bb8630d2c4 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:04.807309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7fc2e-2789-4ec4-8c0d-25bb8630d2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:04.807673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7fc2e-2789-4ec4-8c0d-25bb8630d2c4 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:04.810021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 154/328] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:04.818280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea52de4-9361-404a-aeb7-89b28249f9c9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:04.818763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea52de4-9361-404a-aeb7-89b28249f9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:04.819077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea52de4-9361-404a-aeb7-89b28249f9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:04.863101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1ea52de4-9361-404a-aeb7-89b28249f9c9 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:04.865261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1ea52de4-9361-404a-aeb7-89b28249f9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:05.016892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:05.106225 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 155/329] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:11:05.124564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355096e0-d826-4adc-b499-73c5f8b2a2ac None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:05.189031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea52de4-9361-404a-aeb7-89b28249f9c9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:05.190148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472637d-bd1a-42bc-bda0-1098da4e5eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:05.192076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472637d-bd1a-42bc-bda0-1098da4e5eb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:05.192856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472637d-bd1a-42bc-bda0-1098da4e5eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:05.193545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9472637d-bd1a-42bc-bda0-1098da4e5eb3 None admin] PATH_INFO: `/v3/system/users/3fd577b795dc4a5e851c7e64887803e0/roles/53b551bc12e84268991ca46bce5da0ec` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:05.195816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472637d-bd1a-42bc-bda0-1098da4e5eb3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3fd577b795dc4a5e851c7e64887803e0, role_id=53b551bc12e84268991ca46bce5da0ec)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:05.198391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9472637d-bd1a-42bc-bda0-1098da4e5eb3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:05.218877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 175/330] 10.222.0.64 () {58 vars in 1334 bytes} [Wed Jul 20 08:11:04 2022] PUT /identity/v3/system/users/3fd577b795dc4a5e851c7e64887803e0/roles/53b551bc12e84268991ca46bce5da0ec => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:11:05.376247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea52de4-9361-404a-aeb7-89b28249f9c9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:05.443951 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395afc9-1b72-4b29-9697-d36c2ecdcb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:05.445371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395afc9-1b72-4b29-9697-d36c2ecdcb8d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:05.445883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395afc9-1b72-4b29-9697-d36c2ecdcb8d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:05.446394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395afc9-1b72-4b29-9697-d36c2ecdcb8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:05.469827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395afc9-1b72-4b29-9697-d36c2ecdcb8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-4395afc9-1b72-4b29-9697-d36c2ecdcb8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:05.483407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:05.485353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:05.486723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 156/331] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:11:05.499154 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472637d-bd1a-42bc-bda0-1098da4e5eb3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:05.647649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:05.763297 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9472637d-bd1a-42bc-bda0-1098da4e5eb3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:05.845005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf3f23-7dab-4819-9da1-dc754857619b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:05.846209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3f23-7dab-4819-9da1-dc754857619b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:05.846794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3f23-7dab-4819-9da1-dc754857619b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:05.847872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3f23-7dab-4819-9da1-dc754857619b None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:05.849396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf3f23-7dab-4819-9da1-dc754857619b None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:05.849962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf3f23-7dab-4819-9da1-dc754857619b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:05.852749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf3f23-7dab-4819-9da1-dc754857619b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:05.875387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 176/332] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:11:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:05.885527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395afc9-1b72-4b29-9697-d36c2ecdcb8d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:05.981422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:06.110924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395afc9-1b72-4b29-9697-d36c2ecdcb8d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:06.172752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7186b48-46a1-4ada-9deb-13bb6a6137fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:06.174142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7186b48-46a1-4ada-9deb-13bb6a6137fd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:06.174434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7186b48-46a1-4ada-9deb-13bb6a6137fd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:06.174754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7186b48-46a1-4ada-9deb-13bb6a6137fd None admin] PATH_INFO: `/v3/users/b493ca31dac24b3289024bd29e53e249` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:06.179642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7186b48-46a1-4ada-9deb-13bb6a6137fd None admin] RBAC: Authorizing `identity:get_user(user_id=b493ca31dac24b3289024bd29e53e249)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:06.181356 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7186b48-46a1-4ada-9deb-13bb6a6137fd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:06.183735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 157/333] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:11:05 2022] GET /identity/v3/users/b493ca31dac24b3289024bd29e53e249 => generated 293 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:06.194212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf3f23-7dab-4819-9da1-dc754857619b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:06.395403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf3f23-7dab-4819-9da1-dc754857619b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:06.425741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:06.469546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67d1f3-3284-4618-85b9-b2c907b758f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:06.471191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67d1f3-3284-4618-85b9-b2c907b758f8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:06.472626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67d1f3-3284-4618-85b9-b2c907b758f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:06.473362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67d1f3-3284-4618-85b9-b2c907b758f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:06.475200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67d1f3-3284-4618-85b9-b2c907b758f8 None admin] RBAC: Adding query filter params (role.id=ef48ab9c5b844be59351a260130d3071, user.id=b493ca31dac24b3289024bd29e53e249, scope.system=all) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:06.476014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67d1f3-3284-4618-85b9-b2c907b758f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:06.478782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d67d1f3-3284-4618-85b9-b2c907b758f8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:06.504327 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 177/334] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Jul 20 08:11:06 2022] GET /identity/v3/role_assignments?user.id=b493ca31dac24b3289024bd29e53e249&scope.system=all&role.id=ef48ab9c5b844be59351a260130d3071 => generated 510 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:07.967584 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780926dd-4017-4039-b957-d123f08f031b None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:07.968374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780926dd-4017-4039-b957-d123f08f031b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:07.968508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780926dd-4017-4039-b957-d123f08f031b None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:07.971300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 158/335] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:07.985817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dae9b-a005-44c8-8600-b016f0b8f4ea None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:07.987243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dae9b-a005-44c8-8600-b016f0b8f4ea None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:07.988545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dae9b-a005-44c8-8600-b016f0b8f4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:08.030089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-228dae9b-a005-44c8-8600-b016f0b8f4ea None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:08.030507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-228dae9b-a005-44c8-8600-b016f0b8f4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:08.068203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25843b2-6a73-4934-9f1e-620425b18f5a None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:08.068718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25843b2-6a73-4934-9f1e-620425b18f5a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:08.069335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25843b2-6a73-4934-9f1e-620425b18f5a None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:08.072042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 159/336] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:08.084774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ca77-5126-4ff1-8b7c-39d21ddf5b27 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:08.085297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ca77-5126-4ff1-8b7c-39d21ddf5b27 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:08.085798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51ca77-5126-4ff1-8b7c-39d21ddf5b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:08.156624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7f51ca77-5126-4ff1-8b7c-39d21ddf5b27 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:08.158710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7f51ca77-5126-4ff1-8b7c-39d21ddf5b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:08.267141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 178/337] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:11:08.381087 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 160/338] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:11:08.384996 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228dae9b-a005-44c8-8600-b016f0b8f4ea None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:08.452432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51ca77-5126-4ff1-8b7c-39d21ddf5b27 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:08.607208 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:08.682719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228dae9b-a005-44c8-8600-b016f0b8f4ea None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:08.701398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:08.780956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f841e24-f455-4720-86ee-ddc3de092dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:08.782448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f841e24-f455-4720-86ee-ddc3de092dee None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:08.783030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f841e24-f455-4720-86ee-ddc3de092dee None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:08.783547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f841e24-f455-4720-86ee-ddc3de092dee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:08.797236 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51ca77-5126-4ff1-8b7c-39d21ddf5b27 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:08.810519 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f841e24-f455-4720-86ee-ddc3de092dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5f841e24-f455-4720-86ee-ddc3de092dee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:08.835349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:08.836398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:08.837619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:08.841351 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 179/339] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:11:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 454 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:08.852422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f841e24-f455-4720-86ee-ddc3de092dee None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:08.900180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90d88f-14b9-4743-b2b8-26e4a212a1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:08.901491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90d88f-14b9-4743-b2b8-26e4a212a1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:08.902021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90d88f-14b9-4743-b2b8-26e4a212a1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:08.902517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f90d88f-14b9-4743-b2b8-26e4a212a1f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:08.934370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f90d88f-14b9-4743-b2b8-26e4a212a1f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-3f90d88f-14b9-4743-b2b8-26e4a212a1f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:08.949021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:08.950197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:08.951021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:08.955209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 161/340] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 504 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:08.963911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90d88f-14b9-4743-b2b8-26e4a212a1f7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:09.125451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f841e24-f455-4720-86ee-ddc3de092dee None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:09.151245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:09.159034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f90d88f-14b9-4743-b2b8-26e4a212a1f7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:09.212313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e98525-3726-450e-ba3c-f8fe73c28bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:09.213047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e98525-3726-450e-ba3c-f8fe73c28bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:09.213334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e98525-3726-450e-ba3c-f8fe73c28bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:09.213615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e98525-3726-450e-ba3c-f8fe73c28bf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:09.214564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e98525-3726-450e-ba3c-f8fe73c28bf9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:09.214903 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e98525-3726-450e-ba3c-f8fe73c28bf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:09.216586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e98525-3726-450e-ba3c-f8fe73c28bf9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:09.240518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 180/341] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:09.241196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb5232-6f5e-48aa-995b-421789ca63db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:09.241999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb5232-6f5e-48aa-995b-421789ca63db None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:09.242323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb5232-6f5e-48aa-995b-421789ca63db None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:09.242697 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb5232-6f5e-48aa-995b-421789ca63db None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:09.243620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb5232-6f5e-48aa-995b-421789ca63db None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:09.243956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb5232-6f5e-48aa-995b-421789ca63db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:09.245493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb5232-6f5e-48aa-995b-421789ca63db None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:09.261842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e98525-3726-450e-ba3c-f8fe73c28bf9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:09.269653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 162/342] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:11:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:09.279283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb5232-6f5e-48aa-995b-421789ca63db None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:09.332631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:09.494619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e98525-3726-450e-ba3c-f8fe73c28bf9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:09.506297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb5232-6f5e-48aa-995b-421789ca63db None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:09.579988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7cbde-0d08-47e3-a889-143d83be73ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:09.581600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7cbde-0d08-47e3-a889-143d83be73ab None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:09.582144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7cbde-0d08-47e3-a889-143d83be73ab None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:09.586917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7cbde-0d08-47e3-a889-143d83be73ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:09.619449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31728339-c975-4725-aa75-b0d3149a2cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:09.620447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728339-c975-4725-aa75-b0d3149a2cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:09.620918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728339-c975-4725-aa75-b0d3149a2cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:09.621426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31728339-c975-4725-aa75-b0d3149a2cf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:09.694790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:09.776014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7cbde-0d08-47e3-a889-143d83be73ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:09.779473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7cbde-0d08-47e3-a889-143d83be73ab None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:09.794637 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31728339-c975-4725-aa75-b0d3149a2cf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:09.801558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31728339-c975-4725-aa75-b0d3149a2cf5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:09.883235 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:09.997553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 163/343] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 08:11:10.009077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31728339-c975-4725-aa75-b0d3149a2cf5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:10.149291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 181/344] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 888 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 08:11:10.177062 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7cbde-0d08-47e3-a889-143d83be73ab None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:10.211754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:10.359213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31728339-c975-4725-aa75-b0d3149a2cf5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:10.416738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7cbde-0d08-47e3-a889-143d83be73ab None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:10.434353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857299bb-9a37-4b12-a57f-3f9cabf70cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:10.437183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857299bb-9a37-4b12-a57f-3f9cabf70cdd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:10.437966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857299bb-9a37-4b12-a57f-3f9cabf70cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:10.438681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857299bb-9a37-4b12-a57f-3f9cabf70cdd None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:10.441773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857299bb-9a37-4b12-a57f-3f9cabf70cdd None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:10.444759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857299bb-9a37-4b12-a57f-3f9cabf70cdd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:10.448841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 164/345] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:10 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:10.451551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f6ac8-16e6-4ce0-95b5-ea453b5de89d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:10.452280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f6ac8-16e6-4ce0-95b5-ea453b5de89d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:10.452799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:10.453208 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f6ac8-16e6-4ce0-95b5-ea453b5de89d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:10.456453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 165/346] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:10.458397 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f6ac8-16e6-4ce0-95b5-ea453b5de89d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:10.490293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bfa931-1b34-4d58-a381-0b346c5dfc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:10.492236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfa931-1b34-4d58-a381-0b346c5dfc0b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:10.492955 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfa931-1b34-4d58-a381-0b346c5dfc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:10.493634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfa931-1b34-4d58-a381-0b346c5dfc0b None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:10.496777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfa931-1b34-4d58-a381-0b346c5dfc0b None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:10.502264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bfa931-1b34-4d58-a381-0b346c5dfc0b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:10.506607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 182/347] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:10 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:10.509419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314a45-560f-49b0-a053-9f5d0daefa90 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:10.510031 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314a45-560f-49b0-a053-9f5d0daefa90 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:10.510452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f314a45-560f-49b0-a053-9f5d0daefa90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:10.548407 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4f314a45-560f-49b0-a053-9f5d0daefa90 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:10.551277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4f314a45-560f-49b0-a053-9f5d0daefa90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:10.704524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 183/348] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:10.707391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7e1e-0f3d-45ec-95b7-42dd4d3067f2 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:10.708251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7e1e-0f3d-45ec-95b7-42dd4d3067f2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:10.708708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d7e1e-0f3d-45ec-95b7-42dd4d3067f2 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:10.711135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 184/349] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:10.711776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f6ac8-16e6-4ce0-95b5-ea453b5de89d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:10.712472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d7e1e-0f3d-45ec-95b7-42dd4d3067f2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:10.785925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:10.796156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7197d-716b-407f-b7f7-8e2ec9b6f9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:10.797717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7197d-716b-407f-b7f7-8e2ec9b6f9ee None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:10.798484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7197d-716b-407f-b7f7-8e2ec9b6f9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:10.799073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7197d-716b-407f-b7f7-8e2ec9b6f9ee None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:10.817264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7197d-716b-407f-b7f7-8e2ec9b6f9ee None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:10.820893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc7197d-716b-407f-b7f7-8e2ec9b6f9ee None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:10.824636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 166/350] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:10 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:10.827348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a61d3-7263-4db7-b427-f9430eaec6cd None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:10.828224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a61d3-7263-4db7-b427-f9430eaec6cd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:10.829022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a61d3-7263-4db7-b427-f9430eaec6cd None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:10.833126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 167/351] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:10.836553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f71e35-fc57-46d6-a15f-ddc15ae791b9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:10.837339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f71e35-fc57-46d6-a15f-ddc15ae791b9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:10.838197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f71e35-fc57-46d6-a15f-ddc15ae791b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:10.877530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a7f71e35-fc57-46d6-a15f-ddc15ae791b9 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:10.880967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a7f71e35-fc57-46d6-a15f-ddc15ae791b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:10.976598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d7e1e-0f3d-45ec-95b7-42dd4d3067f2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:11.039280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:11.050098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 168/352] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:11.051447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f71e35-fc57-46d6-a15f-ddc15ae791b9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:11.082420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3219b78-173f-4576-8f66-7800e8d981ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:11.084304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3219b78-173f-4576-8f66-7800e8d981ac None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:11.085044 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3219b78-173f-4576-8f66-7800e8d981ac None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:11.085736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3219b78-173f-4576-8f66-7800e8d981ac None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:11.088670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3219b78-173f-4576-8f66-7800e8d981ac None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:11.093881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3219b78-173f-4576-8f66-7800e8d981ac None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:11.099044 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 185/353] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:10 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:11.101265 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3219b78-173f-4576-8f66-7800e8d981ac None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:11.273268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f71e35-fc57-46d6-a15f-ddc15ae791b9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:11.319030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:11.334155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3219b78-173f-4576-8f66-7800e8d981ac None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:11.343092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da28cb-65c8-4a39-9ede-5e5c308f4f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:11.343092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da28cb-65c8-4a39-9ede-5e5c308f4f4c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:11.343092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da28cb-65c8-4a39-9ede-5e5c308f4f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:11.343092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da28cb-65c8-4a39-9ede-5e5c308f4f4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:11.363462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da28cb-65c8-4a39-9ede-5e5c308f4f4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-02da28cb-65c8-4a39-9ede-5e5c308f4f4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:11.379440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:11.381401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:11.383223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:11.384754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b3dd8-cab6-4490-bda2-a0c177933e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:11.385494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b3dd8-cab6-4490-bda2-a0c177933e12 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:11.385816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b3dd8-cab6-4490-bda2-a0c177933e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:11.386097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b3dd8-cab6-4490-bda2-a0c177933e12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:11.386898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 169/354] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:11.387105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b3dd8-cab6-4490-bda2-a0c177933e12 None admin] RBAC: Adding query filter params (group.id=3aa3b4acf6304cc28f9bc849e2ee5593, role.id=ef48ab9c5b844be59351a260130d3071, scope.project.id=5902af097cc346559cbefdb217f7394c) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:11.387588 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b3dd8-cab6-4490-bda2-a0c177933e12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:11.388923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff1a90-172e-4061-94d2-e0024291fb63 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:11.389437 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff1a90-172e-4061-94d2-e0024291fb63 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:11.389601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36b3dd8-cab6-4490-bda2-a0c177933e12 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:11.390085 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff1a90-172e-4061-94d2-e0024291fb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:11.397091 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 186/355] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Jul 20 08:11:11 2022] GET /identity/v3/role_assignments?group.id=3aa3b4acf6304cc28f9bc849e2ee5593&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=ef48ab9c5b844be59351a260130d3071 => generated 260 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:11.398297 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b3dd8-cab6-4490-bda2-a0c177933e12 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:11.428287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-29ff1a90-172e-4061-94d2-e0024291fb63 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:11.431651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-29ff1a90-172e-4061-94d2-e0024291fb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:11.532328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:11.605250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 170/356] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:11.606259 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff1a90-172e-4061-94d2-e0024291fb63 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:11.630366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b3dd8-cab6-4490-bda2-a0c177933e12 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:11.695158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6363ce-ca11-4184-8a82-478a7ecde1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:11.696522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6363ce-ca11-4184-8a82-478a7ecde1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:11.696828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6363ce-ca11-4184-8a82-478a7ecde1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:11.697171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6363ce-ca11-4184-8a82-478a7ecde1d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:11.719347 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6363ce-ca11-4184-8a82-478a7ecde1d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0b6363ce-ca11-4184-8a82-478a7ecde1d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:11.736525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:11.739458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:11.741056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 187/357] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:11.742373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6363ce-ca11-4184-8a82-478a7ecde1d5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:11.792484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff1a90-172e-4061-94d2-e0024291fb63 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:11.821296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:11.861935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926b985-6df7-486f-b49e-9003210dd00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:11.864180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926b985-6df7-486f-b49e-9003210dd00b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:11.864180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926b985-6df7-486f-b49e-9003210dd00b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:11.864180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926b985-6df7-486f-b49e-9003210dd00b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:11.864985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926b985-6df7-486f-b49e-9003210dd00b None admin] RBAC: Adding query filter params (group.id=3aa3b4acf6304cc28f9bc849e2ee5593, role.id=1d508b4effb546039e87dd273933af16, scope.project.id=5902af097cc346559cbefdb217f7394c) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:11.865305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926b985-6df7-486f-b49e-9003210dd00b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:11.868180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926b985-6df7-486f-b49e-9003210dd00b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:11.876624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 171/358] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Jul 20 08:11:11 2022] GET /identity/v3/role_assignments?group.id=3aa3b4acf6304cc28f9bc849e2ee5593&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=1d508b4effb546039e87dd273933af16 => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:11.878966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a96d2-4dc3-48d4-a741-3220656dc417 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:11.879325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a96d2-4dc3-48d4-a741-3220656dc417 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:11.879621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0a96d2-4dc3-48d4-a741-3220656dc417 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:11.881479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 172/359] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:11.882413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a96d2-4dc3-48d4-a741-3220656dc417 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:11.962037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6363ce-ca11-4184-8a82-478a7ecde1d5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:11.996223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:12.025175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73960015-46c9-4b15-ab4d-a68df66eefc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:12.032232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73960015-46c9-4b15-ab4d-a68df66eefc6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:12.032232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73960015-46c9-4b15-ab4d-a68df66eefc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:12.032232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73960015-46c9-4b15-ab4d-a68df66eefc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:12.032232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73960015-46c9-4b15-ab4d-a68df66eefc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:12.032232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73960015-46c9-4b15-ab4d-a68df66eefc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:12.032232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73960015-46c9-4b15-ab4d-a68df66eefc6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:12.045143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 188/360] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:11:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:12.046499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73960015-46c9-4b15-ab4d-a68df66eefc6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:12.121263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0a96d2-4dc3-48d4-a741-3220656dc417 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:12.189792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8f2f4-8f04-46cd-b423-9d8f5bd2a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:12.192856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f2f4-8f04-46cd-b423-9d8f5bd2a228 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:12.193622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f2f4-8f04-46cd-b423-9d8f5bd2a228 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:12.194264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f2f4-8f04-46cd-b423-9d8f5bd2a228 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:12.213196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73960015-46c9-4b15-ab4d-a68df66eefc6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:12.222819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8f2f4-8f04-46cd-b423-9d8f5bd2a228 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-26c8f2f4-8f04-46cd-b423-9d8f5bd2a228 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:12.244063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:12.246095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:12.248149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:12.253378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 173/361] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:11:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:12.257135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b683b-2eeb-477c-a301-e6e655b08e53 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:12.257926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b683b-2eeb-477c-a301-e6e655b08e53 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:12.258659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b683b-2eeb-477c-a301-e6e655b08e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:12.293648 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0b6b683b-2eeb-477c-a301-e6e655b08e53 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:12.296633 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0b6b683b-2eeb-477c-a301-e6e655b08e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:12.303653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc88201-686d-4cd0-b80a-207a9244f0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:12.305052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc88201-686d-4cd0-b80a-207a9244f0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:12.305709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc88201-686d-4cd0-b80a-207a9244f0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:12.306370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc88201-686d-4cd0-b80a-207a9244f0d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:12.308096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc88201-686d-4cd0-b80a-207a9244f0d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:12.309678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc88201-686d-4cd0-b80a-207a9244f0d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:12.313407 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc88201-686d-4cd0-b80a-207a9244f0d2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:12.341664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 189/362] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:12.341664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc88201-686d-4cd0-b80a-207a9244f0d2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:12.354434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:12.417380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:12.444824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 174/363] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:12.447000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b683b-2eeb-477c-a301-e6e655b08e53 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:12.504607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc88201-686d-4cd0-b80a-207a9244f0d2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:12.582222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d98fe6-3932-4d7c-968e-e2e91e2e2515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:12.583482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d98fe6-3932-4d7c-968e-e2e91e2e2515 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:12.584011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d98fe6-3932-4d7c-968e-e2e91e2e2515 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:12.584518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d98fe6-3932-4d7c-968e-e2e91e2e2515 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:12.738022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b683b-2eeb-477c-a301-e6e655b08e53 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:12.780363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d98fe6-3932-4d7c-968e-e2e91e2e2515 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:12.783309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d98fe6-3932-4d7c-968e-e2e91e2e2515 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:12.829759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133cd3f-5410-4ba9-9109-b052fd60c40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:12.837471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133cd3f-5410-4ba9-9109-b052fd60c40f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:12.837471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133cd3f-5410-4ba9-9109-b052fd60c40f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:12.837471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133cd3f-5410-4ba9-9109-b052fd60c40f None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:12.837471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133cd3f-5410-4ba9-9109-b052fd60c40f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:12.837471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133cd3f-5410-4ba9-9109-b052fd60c40f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:12.837471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133cd3f-5410-4ba9-9109-b052fd60c40f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:12.854864 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 175/364] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:12.863307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133cd3f-5410-4ba9-9109-b052fd60c40f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:12.888808 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:12.924008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:13.059981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 190/365] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:13.062568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d98fe6-3932-4d7c-968e-e2e91e2e2515 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:13.078976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133cd3f-5410-4ba9-9109-b052fd60c40f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:13.131829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e930e5-62ce-4a0e-9915-314d663feb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:13.132367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e930e5-62ce-4a0e-9915-314d663feb98 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:13.132654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e930e5-62ce-4a0e-9915-314d663feb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:13.132928 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e930e5-62ce-4a0e-9915-314d663feb98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:13.262360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d98fe6-3932-4d7c-968e-e2e91e2e2515 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:13.263456 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e930e5-62ce-4a0e-9915-314d663feb98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:13.270289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e930e5-62ce-4a0e-9915-314d663feb98 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:13.322841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee5e11-5dcf-4db5-90a5-478951a72d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:13.323667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee5e11-5dcf-4db5-90a5-478951a72d23 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:13.323947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee5e11-5dcf-4db5-90a5-478951a72d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:13.324220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee5e11-5dcf-4db5-90a5-478951a72d23 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:13.333665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:13.338869 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee5e11-5dcf-4db5-90a5-478951a72d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:13.340104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-77ee5e11-5dcf-4db5-90a5-478951a72d23 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:13.352256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:13.355007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:13.356633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 191/366] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:13.359131 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee5e11-5dcf-4db5-90a5-478951a72d23 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:13.471774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 176/367] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:13.472995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e930e5-62ce-4a0e-9915-314d663feb98 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:13.494005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee5e11-5dcf-4db5-90a5-478951a72d23 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:13.538888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52a1f4-15e9-4e77-92f3-fdcacc6fa07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:13.539426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52a1f4-15e9-4e77-92f3-fdcacc6fa07d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:13.539825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52a1f4-15e9-4e77-92f3-fdcacc6fa07d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:13.540217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52a1f4-15e9-4e77-92f3-fdcacc6fa07d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:13.678557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e930e5-62ce-4a0e-9915-314d663feb98 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:13.698534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:13.709245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52a1f4-15e9-4e77-92f3-fdcacc6fa07d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:13.712895 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52a1f4-15e9-4e77-92f3-fdcacc6fa07d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:13.761159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:13.765835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f31af-677a-43ee-bd87-34074f6960c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:13.770190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f31af-677a-43ee-bd87-34074f6960c6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:13.771174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f31af-677a-43ee-bd87-34074f6960c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:13.772021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f31af-677a-43ee-bd87-34074f6960c6 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:13.774942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f31af-677a-43ee-bd87-34074f6960c6 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:13.779158 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f31af-677a-43ee-bd87-34074f6960c6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:13.783751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 177/368] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:13 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:13.788181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f31af-677a-43ee-bd87-34074f6960c6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:13.946248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 192/369] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:13.947882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52a1f4-15e9-4e77-92f3-fdcacc6fa07d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:14.004126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f31af-677a-43ee-bd87-34074f6960c6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:14.075386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd6509-8c7b-4459-bde3-f0cdf77dc322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:14.076491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd6509-8c7b-4459-bde3-f0cdf77dc322 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:14.076983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd6509-8c7b-4459-bde3-f0cdf77dc322 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:14.077484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd6509-8c7b-4459-bde3-f0cdf77dc322 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:14.079142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd6509-8c7b-4459-bde3-f0cdf77dc322 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:14.079711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd6509-8c7b-4459-bde3-f0cdf77dc322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:14.082959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd6509-8c7b-4459-bde3-f0cdf77dc322 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:14.101040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 178/370] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:11:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:14.102573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd6509-8c7b-4459-bde3-f0cdf77dc322 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:14.159861 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:14.168606 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52a1f4-15e9-4e77-92f3-fdcacc6fa07d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:14.234766 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:14.240950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d1d65-cc9e-474a-b9b1-d0ce6b93efb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:14.242856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d1d65-cc9e-474a-b9b1-d0ce6b93efb4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:14.243569 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d1d65-cc9e-474a-b9b1-d0ce6b93efb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:14.244527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d1d65-cc9e-474a-b9b1-d0ce6b93efb4 None admin] PATH_INFO: `/v3/projects/10580c2140c64e24a4fb1ff11f27a985` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:14.247075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d1d65-cc9e-474a-b9b1-d0ce6b93efb4 None admin] RBAC: Authorizing `identity:get_project(project_id=10580c2140c64e24a4fb1ff11f27a985)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:14.250192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d1d65-cc9e-474a-b9b1-d0ce6b93efb4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:14.255133 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 193/371] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:13 2022] GET /identity/v3/projects/10580c2140c64e24a4fb1ff11f27a985 => generated 344 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:14.257116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d1d65-cc9e-474a-b9b1-d0ce6b93efb4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:14.323709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd6509-8c7b-4459-bde3-f0cdf77dc322 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:14.396665 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880430de-220a-48bd-aa1d-02d413779a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:14.398263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880430de-220a-48bd-aa1d-02d413779a2c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:14.398862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880430de-220a-48bd-aa1d-02d413779a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:14.399387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880430de-220a-48bd-aa1d-02d413779a2c None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:14.401936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880430de-220a-48bd-aa1d-02d413779a2c None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:14.404947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880430de-220a-48bd-aa1d-02d413779a2c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:14.414616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 179/372] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:14 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:14.414616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880430de-220a-48bd-aa1d-02d413779a2c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:14.498239 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d1d65-cc9e-474a-b9b1-d0ce6b93efb4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:14.583000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcf25f-c791-41f8-bf5e-73715593f1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:14.586339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcf25f-c791-41f8-bf5e-73715593f1ef None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:14.587200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcf25f-c791-41f8-bf5e-73715593f1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:14.587942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcf25f-c791-41f8-bf5e-73715593f1ef None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:14.590696 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fcf25f-c791-41f8-bf5e-73715593f1ef None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:14.594262 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fcf25f-c791-41f8-bf5e-73715593f1ef None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:14.599187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 194/373] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:14 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:14.601203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcf25f-c791-41f8-bf5e-73715593f1ef None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:14.650248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880430de-220a-48bd-aa1d-02d413779a2c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:14.683558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:14.694192 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:14.708570 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ac06f-139d-4af1-9c11-c1596a1481f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:14.709581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ac06f-139d-4af1-9c11-c1596a1481f5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:14.709891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ac06f-139d-4af1-9c11-c1596a1481f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:14.710175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ac06f-139d-4af1-9c11-c1596a1481f5 None admin] PATH_INFO: `/v3/users/3fd577b795dc4a5e851c7e64887803e0` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:14.711608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ac06f-139d-4af1-9c11-c1596a1481f5 None admin] RBAC: Authorizing `identity:get_user(user_id=3fd577b795dc4a5e851c7e64887803e0)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:14.713461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ac06f-139d-4af1-9c11-c1596a1481f5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:14.715637 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 180/374] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:11:14 2022] GET /identity/v3/users/3fd577b795dc4a5e851c7e64887803e0 => generated 293 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:14.716465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ac06f-139d-4af1-9c11-c1596a1481f5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:14.827642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcf25f-c791-41f8-bf5e-73715593f1ef None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:14.875898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ac06f-139d-4af1-9c11-c1596a1481f5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:14.886311 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc1603-d4e5-407a-aa8a-9381809ce407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:14.888138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc1603-d4e5-407a-aa8a-9381809ce407 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:14.888893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc1603-d4e5-407a-aa8a-9381809ce407 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:14.889645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc1603-d4e5-407a-aa8a-9381809ce407 None admin] PATH_INFO: `/v3/groups/ff0555320df4412e8ea23f2ca97b10d0` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:14.911959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc1603-d4e5-407a-aa8a-9381809ce407 None admin] RBAC: Authorizing `identity:get_group(group_id=ff0555320df4412e8ea23f2ca97b10d0)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:14.913163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc1603-d4e5-407a-aa8a-9381809ce407 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:14.922896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 195/375] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:14 2022] GET /identity/v3/groups/ff0555320df4412e8ea23f2ca97b10d0 => generated 229 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:14.922896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc1603-d4e5-407a-aa8a-9381809ce407 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:14.957854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1383e8-9c2e-456c-81c1-2986710c45bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:14.958936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1383e8-9c2e-456c-81c1-2986710c45bb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:14.959407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1383e8-9c2e-456c-81c1-2986710c45bb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:14.959846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1383e8-9c2e-456c-81c1-2986710c45bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:14.960974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1383e8-9c2e-456c-81c1-2986710c45bb None admin] RBAC: Adding query filter params (group.id=3aa3b4acf6304cc28f9bc849e2ee5593, role.id=ef48ab9c5b844be59351a260130d3071, scope.project.id=366a6c5b1cb444698289e32b2b46918d) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:14.961498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1383e8-9c2e-456c-81c1-2986710c45bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:14.964054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1383e8-9c2e-456c-81c1-2986710c45bb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:14.973296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 181/376] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Jul 20 08:11:14 2022] GET /identity/v3/role_assignments?group.id=3aa3b4acf6304cc28f9bc849e2ee5593&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=ef48ab9c5b844be59351a260130d3071 => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:14.974703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1383e8-9c2e-456c-81c1-2986710c45bb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:15.102876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:15.142779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:15.149691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc1603-d4e5-407a-aa8a-9381809ce407 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:15.195475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1383e8-9c2e-456c-81c1-2986710c45bb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:15.201501 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bb5b6-98f0-410a-b7aa-afd5202187b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:15.202736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bb5b6-98f0-410a-b7aa-afd5202187b9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:15.203191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bb5b6-98f0-410a-b7aa-afd5202187b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:15.204343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bb5b6-98f0-410a-b7aa-afd5202187b9 None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:15.207770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bb5b6-98f0-410a-b7aa-afd5202187b9 None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:15.210268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bb5b6-98f0-410a-b7aa-afd5202187b9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:15.213117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 196/377] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:14 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:15.215112 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66912ea4-eeb4-43ce-9e38-b1eb36cc7072 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:15.215755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66912ea4-eeb4-43ce-9e38-b1eb36cc7072 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:15.216481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66912ea4-eeb4-43ce-9e38-b1eb36cc7072 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:15.224594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 197/378] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:15.226692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66912ea4-eeb4-43ce-9e38-b1eb36cc7072 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:15.256690 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72395427-9dd5-47c5-9481-65a2d7333d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:15.257661 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72395427-9dd5-47c5-9481-65a2d7333d0b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:15.258138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72395427-9dd5-47c5-9481-65a2d7333d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:15.258691 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72395427-9dd5-47c5-9481-65a2d7333d0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:15.260629 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72395427-9dd5-47c5-9481-65a2d7333d0b None admin] RBAC: Adding query filter params (role.id=53b551bc12e84268991ca46bce5da0ec, user.id=3fd577b795dc4a5e851c7e64887803e0, scope.system=all) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:15.260977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72395427-9dd5-47c5-9481-65a2d7333d0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:15.263853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72395427-9dd5-47c5-9481-65a2d7333d0b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:15.292984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 182/379] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Jul 20 08:11:14 2022] GET /identity/v3/role_assignments?user.id=3fd577b795dc4a5e851c7e64887803e0&scope.system=all&role.id=53b551bc12e84268991ca46bce5da0ec => generated 510 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:15.294506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72395427-9dd5-47c5-9481-65a2d7333d0b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:15.502385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66912ea4-eeb4-43ce-9e38-b1eb36cc7072 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:15.552037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72395427-9dd5-47c5-9481-65a2d7333d0b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:15.577432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c84165c-e7e0-4eea-b540-ed390f95afdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:15.578965 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84165c-e7e0-4eea-b540-ed390f95afdb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:15.579693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84165c-e7e0-4eea-b540-ed390f95afdb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:15.580588 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c84165c-e7e0-4eea-b540-ed390f95afdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:15.583351 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c84165c-e7e0-4eea-b540-ed390f95afdb None admin] RBAC: Adding query filter params (group.id=ff0555320df4412e8ea23f2ca97b10d0, role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=10580c2140c64e24a4fb1ff11f27a985) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:15.584185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c84165c-e7e0-4eea-b540-ed390f95afdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:15.587241 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c84165c-e7e0-4eea-b540-ed390f95afdb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:15.599060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:15.599313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 198/380] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Jul 20 08:11:15 2022] GET /identity/v3/role_assignments?group.id=ff0555320df4412e8ea23f2ca97b10d0&scope.project.id=10580c2140c64e24a4fb1ff11f27a985&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 260 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:15.602341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69a963-2165-4033-a390-260d4206e638 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:15.602876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69a963-2165-4033-a390-260d4206e638 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:15.603476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69a963-2165-4033-a390-260d4206e638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:15.630276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d565de-8579-4aba-a5ac-88a8b466abcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:15.631519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d565de-8579-4aba-a5ac-88a8b466abcd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:15.632042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d565de-8579-4aba-a5ac-88a8b466abcd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:15.632562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d565de-8579-4aba-a5ac-88a8b466abcd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:15.634279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d565de-8579-4aba-a5ac-88a8b466abcd None admin] RBAC: Adding query filter params (group.id=3aa3b4acf6304cc28f9bc849e2ee5593, role.id=1d508b4effb546039e87dd273933af16, scope.project.id=366a6c5b1cb444698289e32b2b46918d) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:15.634459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0f69a963-2165-4033-a390-260d4206e638 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:15.634902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d565de-8579-4aba-a5ac-88a8b466abcd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:15.636644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0f69a963-2165-4033-a390-260d4206e638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:15.637902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d565de-8579-4aba-a5ac-88a8b466abcd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:15.650277 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 183/381] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Jul 20 08:11:15 2022] GET /identity/v3/role_assignments?group.id=3aa3b4acf6304cc28f9bc849e2ee5593&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=1d508b4effb546039e87dd273933af16 => generated 260 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:15.657278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da439ed9-4e15-4b1a-9ae7-f3ecc4ecd557 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:15.657786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da439ed9-4e15-4b1a-9ae7-f3ecc4ecd557 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:15.658529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da439ed9-4e15-4b1a-9ae7-f3ecc4ecd557 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:15.667258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 184/382] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:15 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:15.677736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56335e00-ecda-48c1-bc79-6572be29fdf5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:15.678270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56335e00-ecda-48c1-bc79-6572be29fdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:15.678837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56335e00-ecda-48c1-bc79-6572be29fdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:15.694794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:15.740395 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-56335e00-ecda-48c1-bc79-6572be29fdf5 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:15.746901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-56335e00-ecda-48c1-bc79-6572be29fdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:15.794846 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 199/383] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:15.869442 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69a963-2165-4033-a390-260d4206e638 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:15.954233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 185/384] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:11:16.020065 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56335e00-ecda-48c1-bc79-6572be29fdf5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:16.169647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69a963-2165-4033-a390-260d4206e638 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:16.240002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5f81f-b1d6-4a3a-86ea-0b2709d421f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:16.241447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5f81f-b1d6-4a3a-86ea-0b2709d421f9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:16.241918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5f81f-b1d6-4a3a-86ea-0b2709d421f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:16.242383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5f81f-b1d6-4a3a-86ea-0b2709d421f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:16.263134 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5f81f-b1d6-4a3a-86ea-0b2709d421f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-b9d5f81f-b1d6-4a3a-86ea-0b2709d421f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:16.277511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:16.278851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:16.279767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:16.280846 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 200/385] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:16.289927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5f81f-b1d6-4a3a-86ea-0b2709d421f9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:16.290834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56335e00-ecda-48c1-bc79-6572be29fdf5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:16.315319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:16.340969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:16.396003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43279309-9287-4359-838b-3e0759fb486d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:16.397248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43279309-9287-4359-838b-3e0759fb486d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:16.397789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43279309-9287-4359-838b-3e0759fb486d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:16.398273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43279309-9287-4359-838b-3e0759fb486d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:16.435856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43279309-9287-4359-838b-3e0759fb486d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-43279309-9287-4359-838b-3e0759fb486d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:16.450611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:16.451562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:16.452321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:16.455309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 186/386] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:11:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:11:16.462482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43279309-9287-4359-838b-3e0759fb486d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:16.501014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5f81f-b1d6-4a3a-86ea-0b2709d421f9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:16.576909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d2ac1-9291-4308-8b32-0ec7ca782037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:16.578088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2ac1-9291-4308-8b32-0ec7ca782037 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:16.578547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2ac1-9291-4308-8b32-0ec7ca782037 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:16.579134 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d2ac1-9291-4308-8b32-0ec7ca782037 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:16.580599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d2ac1-9291-4308-8b32-0ec7ca782037 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:16.581737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d2ac1-9291-4308-8b32-0ec7ca782037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:16.584481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88d2ac1-9291-4308-8b32-0ec7ca782037 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:16.610717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 201/387] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:11:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:16.613894 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d2ac1-9291-4308-8b32-0ec7ca782037 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:16.786965 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:16.833064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43279309-9287-4359-838b-3e0759fb486d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:16.857065 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d2ac1-9291-4308-8b32-0ec7ca782037 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:16.962781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a8c5e-233e-4ca2-8eae-b1b3d4f7d0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:16.963644 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a8c5e-233e-4ca2-8eae-b1b3d4f7d0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:16.963932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a8c5e-233e-4ca2-8eae-b1b3d4f7d0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:16.964207 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a8c5e-233e-4ca2-8eae-b1b3d4f7d0d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:16.965438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a8c5e-233e-4ca2-8eae-b1b3d4f7d0d8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:16.965995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a8c5e-233e-4ca2-8eae-b1b3d4f7d0d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:16.968421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a8c5e-233e-4ca2-8eae-b1b3d4f7d0d8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:16.992311 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c9e7-019d-46ff-8d1e-22c89cd0c877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:16.993914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c9e7-019d-46ff-8d1e-22c89cd0c877 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:16.994432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c9e7-019d-46ff-8d1e-22c89cd0c877 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:16.994979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c9e7-019d-46ff-8d1e-22c89cd0c877 None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:16.997261 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58c9e7-019d-46ff-8d1e-22c89cd0c877 None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:16.999658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58c9e7-019d-46ff-8d1e-22c89cd0c877 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:17.007522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 202/388] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:16 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:17.007767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 187/389] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 543 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:17.010285 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c9e7-019d-46ff-8d1e-22c89cd0c877 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:17.035532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a8c5e-233e-4ca2-8eae-b1b3d4f7d0d8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:17.139709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c9e7-019d-46ff-8d1e-22c89cd0c877 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:17.189471 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941db13-3686-4d56-ab8d-30228e625266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:17.191550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941db13-3686-4d56-ab8d-30228e625266 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:17.192154 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941db13-3686-4d56-ab8d-30228e625266 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:17.192664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941db13-3686-4d56-ab8d-30228e625266 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:17.195286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941db13-3686-4d56-ab8d-30228e625266 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:17.198816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2941db13-3686-4d56-ab8d-30228e625266 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:17.199767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:17.205224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 203/390] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:17 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:17.216071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941db13-3686-4d56-ab8d-30228e625266 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:17.287170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:17.422639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941db13-3686-4d56-ab8d-30228e625266 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:17.473238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91534f2e-188b-43be-b078-1283feaa652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:17.475516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534f2e-188b-43be-b078-1283feaa652b None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:17.476327 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534f2e-188b-43be-b078-1283feaa652b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:17.477110 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91534f2e-188b-43be-b078-1283feaa652b None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c/groups/3aa3b4acf6304cc28f9bc849e2ee5593/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:17.481398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91534f2e-188b-43be-b078-1283feaa652b None admin] RBAC: Authorizing `identity:create_grant(project_id=5902af097cc346559cbefdb217f7394c, group_id=3aa3b4acf6304cc28f9bc849e2ee5593, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:17.484725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91534f2e-188b-43be-b078-1283feaa652b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:17.484979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a8c5e-233e-4ca2-8eae-b1b3d4f7d0d8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:17.509113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 204/391] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Jul 20 08:11:17 2022] PUT /identity/v3/projects/5902af097cc346559cbefdb217f7394c/groups/3aa3b4acf6304cc28f9bc849e2ee5593/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:11:17.564901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b8c40-07c7-4a38-b359-e62423497032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:17.566476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b8c40-07c7-4a38-b359-e62423497032 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:17.566795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b8c40-07c7-4a38-b359-e62423497032 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:17.567058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b8c40-07c7-4a38-b359-e62423497032 None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:17.572066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b8c40-07c7-4a38-b359-e62423497032 None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:17.575076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b8c40-07c7-4a38-b359-e62423497032 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:17.578284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 188/392] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:17 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 547 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:17.586147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91534f2e-188b-43be-b078-1283feaa652b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:17.755265 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:17.794760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91534f2e-188b-43be-b078-1283feaa652b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:17.866493 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7ac4a-cdc2-4f9b-900e-78b98305e371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:17.868274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7ac4a-cdc2-4f9b-900e-78b98305e371 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:17.869127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7ac4a-cdc2-4f9b-900e-78b98305e371 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:17.869914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7ac4a-cdc2-4f9b-900e-78b98305e371 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:17.872465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7ac4a-cdc2-4f9b-900e-78b98305e371 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:17.875519 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7ac4a-cdc2-4f9b-900e-78b98305e371 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:17.880481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 205/393] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:17 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:17.893775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b8c40-07c7-4a38-b359-e62423497032 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:18.047081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:18.051366 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b8c40-07c7-4a38-b359-e62423497032 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:18.117582 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f505a-405b-41b4-b89d-1b2867488f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:18.119186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f505a-405b-41b4-b89d-1b2867488f00 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:18.119653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f505a-405b-41b4-b89d-1b2867488f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:18.120139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f505a-405b-41b4-b89d-1b2867488f00 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c/groups/3aa3b4acf6304cc28f9bc849e2ee5593/roles/1d508b4effb546039e87dd273933af16` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:18.123824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f505a-405b-41b4-b89d-1b2867488f00 None admin] RBAC: Authorizing `identity:create_grant(project_id=5902af097cc346559cbefdb217f7394c, group_id=3aa3b4acf6304cc28f9bc849e2ee5593, role_id=1d508b4effb546039e87dd273933af16)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:18.126729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f505a-405b-41b4-b89d-1b2867488f00 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:18.160204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 189/394] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Jul 20 08:11:17 2022] PUT /identity/v3/projects/5902af097cc346559cbefdb217f7394c/groups/3aa3b4acf6304cc28f9bc849e2ee5593/roles/1d508b4effb546039e87dd273933af16 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:11:18.703880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33100-2732-4a7e-8785-d49ed11bf32f None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:18.704719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33100-2732-4a7e-8785-d49ed11bf32f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:18.704719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f33100-2732-4a7e-8785-d49ed11bf32f None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:18.708292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 206/395] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:18.716710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29267121-6932-47fd-b063-459ee16759bd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:18.717044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29267121-6932-47fd-b063-459ee16759bd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:18.717309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29267121-6932-47fd-b063-459ee16759bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:18.743902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-29267121-6932-47fd-b063-459ee16759bd None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:18.745142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-29267121-6932-47fd-b063-459ee16759bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:18.878124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 190/396] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:18.947853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33100-2732-4a7e-8785-d49ed11bf32f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:19.124678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f33100-2732-4a7e-8785-d49ed11bf32f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:19.184460 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d84567-e02f-4586-9aec-078f04dec068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:19.185284 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d84567-e02f-4586-9aec-078f04dec068 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:19.185616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d84567-e02f-4586-9aec-078f04dec068 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:19.186185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d84567-e02f-4586-9aec-078f04dec068 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:19.208788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572ceb8-a902-4dac-8720-ee7191c0af06 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:19.210611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572ceb8-a902-4dac-8720-ee7191c0af06 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:19.211178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572ceb8-a902-4dac-8720-ee7191c0af06 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:19.214621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 191/397] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:19.219640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d84567-e02f-4586-9aec-078f04dec068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:19.225211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd06ab-d18f-4a32-8338-f5b5528a70e1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:19.226136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd06ab-d18f-4a32-8338-f5b5528a70e1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:19.226963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd06ab-d18f-4a32-8338-f5b5528a70e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-30d84567-e02f-4586-9aec-078f04dec068 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:19.228596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:19.230358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:19.231439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:19.234853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 207/398] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:11:19.246233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d84567-e02f-4586-9aec-078f04dec068 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:19.280382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:19.295908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-aadd06ab-d18f-4a32-8338-f5b5528a70e1 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:19.298141 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aadd06ab-d18f-4a32-8338-f5b5528a70e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:19.500610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 192/399] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:19.553502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd06ab-d18f-4a32-8338-f5b5528a70e1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:19.555967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d84567-e02f-4586-9aec-078f04dec068 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:19.598411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:19.631737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcf2b5-da4f-4cee-92ec-9673c9bf9612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:19.633309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcf2b5-da4f-4cee-92ec-9673c9bf9612 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:19.634046 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcf2b5-da4f-4cee-92ec-9673c9bf9612 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:19.634790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcf2b5-da4f-4cee-92ec-9673c9bf9612 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:19.636442 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcf2b5-da4f-4cee-92ec-9673c9bf9612 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:19.637257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcf2b5-da4f-4cee-92ec-9673c9bf9612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:19.640277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcf2b5-da4f-4cee-92ec-9673c9bf9612 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:19.657172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 208/400] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:11:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:19.660888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2da5d4-fab6-498f-9789-075459ce8d98 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:19.661687 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2da5d4-fab6-498f-9789-075459ce8d98 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:19.662403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2da5d4-fab6-498f-9789-075459ce8d98 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:19.665806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 209/401] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:19.671747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2da5d4-fab6-498f-9789-075459ce8d98 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:19.760896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd06ab-d18f-4a32-8338-f5b5528a70e1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:19.812334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:19.819366 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afbd53-4df8-40cb-a252-e54a87a1aec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:19.821267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afbd53-4df8-40cb-a252-e54a87a1aec4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:19.822055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afbd53-4df8-40cb-a252-e54a87a1aec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:19.822783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afbd53-4df8-40cb-a252-e54a87a1aec4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:19.847007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afbd53-4df8-40cb-a252-e54a87a1aec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:19.862967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2da5d4-fab6-498f-9789-075459ce8d98 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e4afbd53-4df8-40cb-a252-e54a87a1aec4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:19.865746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:19.867727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:11:19.869614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:19.874531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 193/402] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:11:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:19.877444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05a077-0715-47e3-967e-47d3371b8774 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:19.879268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05a077-0715-47e3-967e-47d3371b8774 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:19.880505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05a077-0715-47e3-967e-47d3371b8774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:19.912782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e8750-a726-4264-8598-cfaa77861a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:19.913798 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e8750-a726-4264-8598-cfaa77861a4d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:19.914100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e8750-a726-4264-8598-cfaa77861a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:19.914364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e8750-a726-4264-8598-cfaa77861a4d None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:19.916277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e8750-a726-4264-8598-cfaa77861a4d None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:19.917897 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9c05a077-0715-47e3-967e-47d3371b8774 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:19.918085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e8750-a726-4264-8598-cfaa77861a4d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:19.926632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 210/403] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:19 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:19.926632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e8750-a726-4264-8598-cfaa77861a4d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:19.927035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9c05a077-0715-47e3-967e-47d3371b8774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:20.082159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 194/404] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:20.084346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05a077-0715-47e3-967e-47d3371b8774 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:20.121544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:20.123941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e8750-a726-4264-8598-cfaa77861a4d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:20.191337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c640d-ce26-440c-92fd-edba28cf47c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:20.192034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c640d-ce26-440c-92fd-edba28cf47c5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:20.192358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c640d-ce26-440c-92fd-edba28cf47c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:20.192637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c640d-ce26-440c-92fd-edba28cf47c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:20.193415 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c640d-ce26-440c-92fd-edba28cf47c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:20.193748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c640d-ce26-440c-92fd-edba28cf47c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:20.195987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c640d-ce26-440c-92fd-edba28cf47c5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:20.200134 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:20.215173 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 211/405] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:20.218607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c640d-ce26-440c-92fd-edba28cf47c5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:20.327788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05a077-0715-47e3-967e-47d3371b8774 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:20.394635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4de96-50d7-4f70-bb86-a0d82c27e450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:20.395992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4de96-50d7-4f70-bb86-a0d82c27e450 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:20.396471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4de96-50d7-4f70-bb86-a0d82c27e450 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:20.396933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4de96-50d7-4f70-bb86-a0d82c27e450 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:20.399215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4de96-50d7-4f70-bb86-a0d82c27e450 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:20.401868 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4de96-50d7-4f70-bb86-a0d82c27e450 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:20.406206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c640d-ce26-440c-92fd-edba28cf47c5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:20.411511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 195/406] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:20 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:20.413256 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4de96-50d7-4f70-bb86-a0d82c27e450 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:20.486571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8d337-77e1-4f79-a57e-3e9475268c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:20.488967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8d337-77e1-4f79-a57e-3e9475268c89 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:20.489727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8d337-77e1-4f79-a57e-3e9475268c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:20.490395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8d337-77e1-4f79-a57e-3e9475268c89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:20.515320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8d337-77e1-4f79-a57e-3e9475268c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-93c8d337-77e1-4f79-a57e-3e9475268c89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:20.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:20.531586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:20.533138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:20.538775 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 212/407] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:11:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:20.540823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8d337-77e1-4f79-a57e-3e9475268c89 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:20.595054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:20.647017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4de96-50d7-4f70-bb86-a0d82c27e450 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:20.660035 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:20.715592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c172160-bf7b-40c3-9d42-047c1f29e596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:20.717332 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c172160-bf7b-40c3-9d42-047c1f29e596 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:20.717830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c172160-bf7b-40c3-9d42-047c1f29e596 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:20.718337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c172160-bf7b-40c3-9d42-047c1f29e596 None admin] PATH_INFO: `/v3/groups/ff0555320df4412e8ea23f2ca97b10d0` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:20.720749 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c172160-bf7b-40c3-9d42-047c1f29e596 None admin] RBAC: Authorizing `identity:get_group(group_id=ff0555320df4412e8ea23f2ca97b10d0)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:20.724656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c172160-bf7b-40c3-9d42-047c1f29e596 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:20.728491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 196/408] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:20 2022] GET /identity/v3/groups/ff0555320df4412e8ea23f2ca97b10d0 => generated 229 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:20.730305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c172160-bf7b-40c3-9d42-047c1f29e596 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:20.762077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8d337-77e1-4f79-a57e-3e9475268c89 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:20.841721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15ea15-6703-4f8b-9adb-5920f7149aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:20.843445 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ea15-6703-4f8b-9adb-5920f7149aee None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:20.843946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ea15-6703-4f8b-9adb-5920f7149aee None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:20.844445 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15ea15-6703-4f8b-9adb-5920f7149aee None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d/groups/3aa3b4acf6304cc28f9bc849e2ee5593/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:20.850291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15ea15-6703-4f8b-9adb-5920f7149aee None admin] RBAC: Authorizing `identity:create_grant(project_id=366a6c5b1cb444698289e32b2b46918d, group_id=3aa3b4acf6304cc28f9bc849e2ee5593, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:20.853318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15ea15-6703-4f8b-9adb-5920f7149aee None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:20.878408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 213/409] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Jul 20 08:11:20 2022] PUT /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d/groups/3aa3b4acf6304cc28f9bc849e2ee5593/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:11:20.880388 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1630ac-6bcc-4860-9a16-9e67dea32375 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:20.880902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1630ac-6bcc-4860-9a16-9e67dea32375 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:20.881528 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1630ac-6bcc-4860-9a16-9e67dea32375 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:20.891119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 214/410] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:20 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:20.892769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1630ac-6bcc-4860-9a16-9e67dea32375 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:20.973211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c172160-bf7b-40c3-9d42-047c1f29e596 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:21.065466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cef20-3c6d-45f3-a7df-2a17e1bb2002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:21.068541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cef20-3c6d-45f3-a7df-2a17e1bb2002 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:21.069474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cef20-3c6d-45f3-a7df-2a17e1bb2002 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:21.070735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cef20-3c6d-45f3-a7df-2a17e1bb2002 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:21.072640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cef20-3c6d-45f3-a7df-2a17e1bb2002 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:21.074017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cef20-3c6d-45f3-a7df-2a17e1bb2002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:21.079053 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cef20-3c6d-45f3-a7df-2a17e1bb2002 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:21.102099 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 197/411] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:21.106433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf06225-b764-4ec3-b287-df1988c3dd9a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:21.107383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf06225-b764-4ec3-b287-df1988c3dd9a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:21.108160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf06225-b764-4ec3-b287-df1988c3dd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:21.146977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3cf06225-b764-4ec3-b287-df1988c3dd9a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:21.149780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3cf06225-b764-4ec3-b287-df1988c3dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:21.150519 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1630ac-6bcc-4860-9a16-9e67dea32375 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:21.155062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:21.210210 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:21.242215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc981b3-a7af-41b6-a970-7d543afc5ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:21.243732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc981b3-a7af-41b6-a970-7d543afc5ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:21.244224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc981b3-a7af-41b6-a970-7d543afc5ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:21.244718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc981b3-a7af-41b6-a970-7d543afc5ea2 None admin] PATH_INFO: `/v3/projects/10580c2140c64e24a4fb1ff11f27a985` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:21.247106 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc981b3-a7af-41b6-a970-7d543afc5ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=10580c2140c64e24a4fb1ff11f27a985)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:21.250169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc981b3-a7af-41b6-a970-7d543afc5ea2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:21.254217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 215/412] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:20 2022] GET /identity/v3/projects/10580c2140c64e24a4fb1ff11f27a985 => generated 344 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:21.255721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc981b3-a7af-41b6-a970-7d543afc5ea2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:21.304409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 198/413] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:21.313013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf06225-b764-4ec3-b287-df1988c3dd9a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:21.453808 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc981b3-a7af-41b6-a970-7d543afc5ea2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:21.527252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf06225-b764-4ec3-b287-df1988c3dd9a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:21.530131 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb198e5-a95a-41e4-bf01-bf688ce25f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:21.531749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb198e5-a95a-41e4-bf01-bf688ce25f6f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:21.533246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb198e5-a95a-41e4-bf01-bf688ce25f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:21.533246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb198e5-a95a-41e4-bf01-bf688ce25f6f None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:21.535260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb198e5-a95a-41e4-bf01-bf688ce25f6f None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:21.538098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb198e5-a95a-41e4-bf01-bf688ce25f6f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:21.549185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 216/414] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:21 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:21.549185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978bb8-8651-4355-bd75-eb9499ce6932 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:21.549185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978bb8-8651-4355-bd75-eb9499ce6932 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:21.549185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978bb8-8651-4355-bd75-eb9499ce6932 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:21.549185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 217/415] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:21.549185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e978bb8-8651-4355-bd75-eb9499ce6932 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:21.612774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea1b29-8183-46a9-ae3b-c7e9a1e1b2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:21.614444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea1b29-8183-46a9-ae3b-c7e9a1e1b2b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:21.614985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea1b29-8183-46a9-ae3b-c7e9a1e1b2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:21.615492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea1b29-8183-46a9-ae3b-c7e9a1e1b2b6 None admin] PATH_INFO: `/v3/projects/10580c2140c64e24a4fb1ff11f27a985/groups/ff0555320df4412e8ea23f2ca97b10d0/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:21.619439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea1b29-8183-46a9-ae3b-c7e9a1e1b2b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=10580c2140c64e24a4fb1ff11f27a985, group_id=ff0555320df4412e8ea23f2ca97b10d0, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:21.622369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea1b29-8183-46a9-ae3b-c7e9a1e1b2b6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:21.642217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:21.658718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 199/416] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Jul 20 08:11:21 2022] PUT /identity/v3/projects/10580c2140c64e24a4fb1ff11f27a985/groups/ff0555320df4412e8ea23f2ca97b10d0/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:11:21.660990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea1b29-8183-46a9-ae3b-c7e9a1e1b2b6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:21.692794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:21.820984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e978bb8-8651-4355-bd75-eb9499ce6932 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:21.880911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea1b29-8183-46a9-ae3b-c7e9a1e1b2b6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:21.895032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadb509-0ae9-4f93-a4d1-0d55e2986501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:21.896893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadb509-0ae9-4f93-a4d1-0d55e2986501 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:21.897355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadb509-0ae9-4f93-a4d1-0d55e2986501 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:21.897736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadb509-0ae9-4f93-a4d1-0d55e2986501 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:21.917670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadb509-0ae9-4f93-a4d1-0d55e2986501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-afadb509-0ae9-4f93-a4d1-0d55e2986501 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:21.936450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:21.938105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:21.939548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:21.941978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 218/417] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:21.944792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7b64-bca6-4757-aaf3-5b84ce4dbe8b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:21.944792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7b64-bca6-4757-aaf3-5b84ce4dbe8b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:21.944792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cf7b64-bca6-4757-aaf3-5b84ce4dbe8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:21.976051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b2cf7b64-bca6-4757-aaf3-5b84ce4dbe8b None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:21.978233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b2cf7b64-bca6-4757-aaf3-5b84ce4dbe8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:21.979314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1765c4-d57e-497e-94e2-c0b06e94875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:21.981284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1765c4-d57e-497e-94e2-c0b06e94875e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:21.982622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1765c4-d57e-497e-94e2-c0b06e94875e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:21.985260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1765c4-d57e-497e-94e2-c0b06e94875e None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:21.988347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1765c4-d57e-497e-94e2-c0b06e94875e None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:21.991273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1765c4-d57e-497e-94e2-c0b06e94875e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:22.002378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 200/418] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:21 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:22.002378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1765c4-d57e-497e-94e2-c0b06e94875e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:22.145578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 219/419] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:22.146216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:22.148421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf7b64-bca6-4757-aaf3-5b84ce4dbe8b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:22.245015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1765c4-d57e-497e-94e2-c0b06e94875e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:22.246515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:22.314419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b45fef-40a3-4cb4-96c0-47b013b48020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:22.316407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b45fef-40a3-4cb4-96c0-47b013b48020 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:22.317001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b45fef-40a3-4cb4-96c0-47b013b48020 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:22.317799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b45fef-40a3-4cb4-96c0-47b013b48020 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:22.319812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b45fef-40a3-4cb4-96c0-47b013b48020 None admin] RBAC: Adding query filter params (name=member) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:22.320762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b45fef-40a3-4cb4-96c0-47b013b48020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:22.322929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b45fef-40a3-4cb4-96c0-47b013b48020 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:22.342642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 201/420] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:11:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:22.349046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b45fef-40a3-4cb4-96c0-47b013b48020 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:22.352656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cf7b64-bca6-4757-aaf3-5b84ce4dbe8b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:22.436472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db67b4-6630-41bf-811b-17bc08900e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:22.438481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db67b4-6630-41bf-811b-17bc08900e7f None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:22.439186 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db67b4-6630-41bf-811b-17bc08900e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:22.439873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db67b4-6630-41bf-811b-17bc08900e7f None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d/groups/3aa3b4acf6304cc28f9bc849e2ee5593/roles/1d508b4effb546039e87dd273933af16` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:22.445584 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db67b4-6630-41bf-811b-17bc08900e7f None admin] RBAC: Authorizing `identity:create_grant(project_id=366a6c5b1cb444698289e32b2b46918d, group_id=3aa3b4acf6304cc28f9bc849e2ee5593, role_id=1d508b4effb546039e87dd273933af16)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:22.449853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db67b4-6630-41bf-811b-17bc08900e7f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:22.481654 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 220/421] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Jul 20 08:11:22 2022] PUT /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d/groups/3aa3b4acf6304cc28f9bc849e2ee5593/roles/1d508b4effb546039e87dd273933af16 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:11:22.483981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db67b4-6630-41bf-811b-17bc08900e7f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:22.600958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b45fef-40a3-4cb4-96c0-47b013b48020 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:22.670653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:22.702249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a54991-ebc6-4399-a3a5-347d935a4e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:22.703657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a54991-ebc6-4399-a3a5-347d935a4e05 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:22.704168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a54991-ebc6-4399-a3a5-347d935a4e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:22.704672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a54991-ebc6-4399-a3a5-347d935a4e05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:22.748509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a54991-ebc6-4399-a3a5-347d935a4e05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:22.761904 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db67b4-6630-41bf-811b-17bc08900e7f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-98a54991-ebc6-4399-a3a5-347d935a4e05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:22.765643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:22.767705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:22.770328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:22.774869 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 202/422] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:11:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 421 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:22.783063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:22.811373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a54991-ebc6-4399-a3a5-347d935a4e05 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:22.876868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701bc8f-4d30-4cdc-a7f4-820bdfe35444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:22.882805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701bc8f-4d30-4cdc-a7f4-820bdfe35444 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:22.883658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701bc8f-4d30-4cdc-a7f4-820bdfe35444 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:22.884670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701bc8f-4d30-4cdc-a7f4-820bdfe35444 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:23.085443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a54991-ebc6-4399-a3a5-347d935a4e05 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:23.105888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701bc8f-4d30-4cdc-a7f4-820bdfe35444 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:23.108850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701bc8f-4d30-4cdc-a7f4-820bdfe35444 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:23.231150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fd9d4-ca56-4972-92bd-653bd2cc02b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:23.232372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fd9d4-ca56-4972-92bd-653bd2cc02b9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:23.232898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fd9d4-ca56-4972-92bd-653bd2cc02b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:23.233409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fd9d4-ca56-4972-92bd-653bd2cc02b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:23.234942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fd9d4-ca56-4972-92bd-653bd2cc02b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:23.235768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fd9d4-ca56-4972-92bd-653bd2cc02b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:23.239249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fd9d4-ca56-4972-92bd-653bd2cc02b9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:23.257283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 203/423] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:23.269428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fd9d4-ca56-4972-92bd-653bd2cc02b9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:23.455323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:23.471962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 221/424] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 989 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:23.475582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701bc8f-4d30-4cdc-a7f4-820bdfe35444 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:23.485904 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:23.561887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fd9d4-ca56-4972-92bd-653bd2cc02b9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:23.638700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e409c8-b2af-4e46-a96c-bd7940543a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:23.639894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e409c8-b2af-4e46-a96c-bd7940543a2f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:23.640391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e409c8-b2af-4e46-a96c-bd7940543a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:23.640885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e409c8-b2af-4e46-a96c-bd7940543a2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:23.791549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e409c8-b2af-4e46-a96c-bd7940543a2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:23.794517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e409c8-b2af-4e46-a96c-bd7940543a2f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:23.886113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701bc8f-4d30-4cdc-a7f4-820bdfe35444 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:23.923107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:23.969632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96b004-3793-4149-8b7f-dab42df79bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:23.971977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96b004-3793-4149-8b7f-dab42df79bfd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:23.972962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96b004-3793-4149-8b7f-dab42df79bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:23.973787 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96b004-3793-4149-8b7f-dab42df79bfd None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:23.977487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96b004-3793-4149-8b7f-dab42df79bfd None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:23.980994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96b004-3793-4149-8b7f-dab42df79bfd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:23.985739 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 222/425] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:23 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 511 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:23.995567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96b004-3793-4149-8b7f-dab42df79bfd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:24.088095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 204/426] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 818 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:24.093774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e409c8-b2af-4e46-a96c-bd7940543a2f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:24.271961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:24.350458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96b004-3793-4149-8b7f-dab42df79bfd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:24.432632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ca231-a3f7-4896-9495-7b64137f08e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:24.434796 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ca231-a3f7-4896-9495-7b64137f08e2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:24.435271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ca231-a3f7-4896-9495-7b64137f08e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:24.436409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ca231-a3f7-4896-9495-7b64137f08e2 None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:24.438810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ca231-a3f7-4896-9495-7b64137f08e2 None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:24.439335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e409c8-b2af-4e46-a96c-bd7940543a2f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:24.441503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ca231-a3f7-4896-9495-7b64137f08e2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:24.447193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 223/427] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:23 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:24.449502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a503166-a88d-44c2-8c42-27c88f2ae13c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:24.449942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a503166-a88d-44c2-8c42-27c88f2ae13c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:24.450375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a503166-a88d-44c2-8c42-27c88f2ae13c None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:24.453305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 224/428] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:24.459006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a503166-a88d-44c2-8c42-27c88f2ae13c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:24.526314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71094523-168a-4d74-86a7-546bbe0190d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:24.528218 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71094523-168a-4d74-86a7-546bbe0190d2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:24.528703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71094523-168a-4d74-86a7-546bbe0190d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:24.529190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71094523-168a-4d74-86a7-546bbe0190d2 None admin] PATH_INFO: `/v3/projects/5902af097cc346559cbefdb217f7394c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:24.532576 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71094523-168a-4d74-86a7-546bbe0190d2 None admin] RBAC: Authorizing `identity:get_project(project_id=5902af097cc346559cbefdb217f7394c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:24.536282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71094523-168a-4d74-86a7-546bbe0190d2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:24.541123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 205/429] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:24 2022] GET /identity/v3/projects/5902af097cc346559cbefdb217f7394c => generated 298 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:24.543444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f0d3b-b3e6-44f5-aa25-3fcd033aa8ec None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:24.543938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f0d3b-b3e6-44f5-aa25-3fcd033aa8ec None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:24.544432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f0d3b-b3e6-44f5-aa25-3fcd033aa8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:24.578306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d29f0d3b-b3e6-44f5-aa25-3fcd033aa8ec None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:24.581662 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d29f0d3b-b3e6-44f5-aa25-3fcd033aa8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:24.611244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:24.706623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a503166-a88d-44c2-8c42-27c88f2ae13c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:24.727986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 206/430] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:24.729550 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f0d3b-b3e6-44f5-aa25-3fcd033aa8ec None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:24.779809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215352e-6e6a-44f1-88a4-1c9fbbcc4fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:24.780999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215352e-6e6a-44f1-88a4-1c9fbbcc4fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:24.781509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215352e-6e6a-44f1-88a4-1c9fbbcc4fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:24.782010 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215352e-6e6a-44f1-88a4-1c9fbbcc4fe5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:24.785163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215352e-6e6a-44f1-88a4-1c9fbbcc4fe5 None admin] RBAC: Adding query filter params (group.id=3aa3b4acf6304cc28f9bc849e2ee5593, role.id=ef48ab9c5b844be59351a260130d3071, scope.project.id=5902af097cc346559cbefdb217f7394c) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:24.785736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215352e-6e6a-44f1-88a4-1c9fbbcc4fe5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:24.788603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d215352e-6e6a-44f1-88a4-1c9fbbcc4fe5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:24.801374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 225/431] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Jul 20 08:11:24 2022] GET /identity/v3/role_assignments?group.id=3aa3b4acf6304cc28f9bc849e2ee5593&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=ef48ab9c5b844be59351a260130d3071 => generated 611 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:24.803422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215352e-6e6a-44f1-88a4-1c9fbbcc4fe5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:24.810064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:24.950642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f0d3b-b3e6-44f5-aa25-3fcd033aa8ec None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:25.009509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036371-b982-4463-971d-6e047db71fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:25.010618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215352e-6e6a-44f1-88a4-1c9fbbcc4fe5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:25.010785 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036371-b982-4463-971d-6e047db71fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:25.011131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036371-b982-4463-971d-6e047db71fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:25.011451 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036371-b982-4463-971d-6e047db71fd6 None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:25.013388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036371-b982-4463-971d-6e047db71fd6 None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:25.015955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036371-b982-4463-971d-6e047db71fd6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:25.019404 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 207/432] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:24 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:25.027173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036371-b982-4463-971d-6e047db71fd6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:25.054912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:25.083778 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8456ba-bfc0-4a08-9b83-83ba95a511e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:25.085592 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8456ba-bfc0-4a08-9b83-83ba95a511e8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:25.086336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8456ba-bfc0-4a08-9b83-83ba95a511e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:25.087128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8456ba-bfc0-4a08-9b83-83ba95a511e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:25.109467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8456ba-bfc0-4a08-9b83-83ba95a511e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ff8456ba-bfc0-4a08-9b83-83ba95a511e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:25.125933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:25.128055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 08:11:25.129609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:25.134244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 226/433] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:25.147906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8456ba-bfc0-4a08-9b83-83ba95a511e8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:25.182835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036371-b982-4463-971d-6e047db71fd6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:25.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58f19e-c9fb-4e23-b478-17df403fcb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:25.241645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58f19e-c9fb-4e23-b478-17df403fcb63 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:25.242002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58f19e-c9fb-4e23-b478-17df403fcb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:25.242283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58f19e-c9fb-4e23-b478-17df403fcb63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:25.243301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58f19e-c9fb-4e23-b478-17df403fcb63 None admin] RBAC: Adding query filter params (group.id=3aa3b4acf6304cc28f9bc849e2ee5593, role.id=1d508b4effb546039e87dd273933af16, scope.project.id=5902af097cc346559cbefdb217f7394c) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:25.244269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58f19e-c9fb-4e23-b478-17df403fcb63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:25.246510 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58f19e-c9fb-4e23-b478-17df403fcb63 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:25.263469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 208/434] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Jul 20 08:11:25 2022] GET /identity/v3/role_assignments?group.id=3aa3b4acf6304cc28f9bc849e2ee5593&scope.project.id=5902af097cc346559cbefdb217f7394c&role.id=1d508b4effb546039e87dd273933af16 => generated 611 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:25.314084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:25.404749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8456ba-bfc0-4a08-9b83-83ba95a511e8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:25.491951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed5e88-e37b-4b8d-8485-55b9d5516650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:25.493688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed5e88-e37b-4b8d-8485-55b9d5516650 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:25.494546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed5e88-e37b-4b8d-8485-55b9d5516650 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:25.495393 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed5e88-e37b-4b8d-8485-55b9d5516650 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:25.497328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed5e88-e37b-4b8d-8485-55b9d5516650 None admin] RBAC: Adding query filter params (name=member) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:25.498293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed5e88-e37b-4b8d-8485-55b9d5516650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:25.501346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed5e88-e37b-4b8d-8485-55b9d5516650 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:25.520950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 227/435] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:11:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:25.530098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58f19e-c9fb-4e23-b478-17df403fcb63 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:25.626171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64165633-e053-4313-877b-d18b639adab7 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:25.626914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64165633-e053-4313-877b-d18b639adab7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:25.627466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64165633-e053-4313-877b-d18b639adab7 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:25.630188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 228/436] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:11:25.639685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f6444-5b53-4ade-98a1-47e05598fa4e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:25.640185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f6444-5b53-4ade-98a1-47e05598fa4e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:25.640691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f6444-5b53-4ade-98a1-47e05598fa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:25.682557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-dc5f6444-5b53-4ade-98a1-47e05598fa4e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:25.685827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dc5f6444-5b53-4ade-98a1-47e05598fa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:25.718262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:25.764554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58f19e-c9fb-4e23-b478-17df403fcb63 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:25.835894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6632e-0d48-4889-9282-2f7dd314ef53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:25.837264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6632e-0d48-4889-9282-2f7dd314ef53 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:25.837908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6632e-0d48-4889-9282-2f7dd314ef53 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:25.838835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6632e-0d48-4889-9282-2f7dd314ef53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:25.860409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 229/437] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:11:25.896943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f6444-5b53-4ade-98a1-47e05598fa4e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:25.995870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6632e-0d48-4889-9282-2f7dd314ef53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:25.998021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6632e-0d48-4889-9282-2f7dd314ef53 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:26.034736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:26.142791 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f6444-5b53-4ade-98a1-47e05598fa4e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:26.178796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:26.225588 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b67f18-6103-415a-9897-3431edd9fecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:26.226450 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 209/438] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:26.227020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b67f18-6103-415a-9897-3431edd9fecf None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:26.227530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b67f18-6103-415a-9897-3431edd9fecf None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:26.228121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b67f18-6103-415a-9897-3431edd9fecf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:26.235372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6632e-0d48-4889-9282-2f7dd314ef53 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:26.252470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b67f18-6103-415a-9897-3431edd9fecf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-76b67f18-6103-415a-9897-3431edd9fecf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:26.270211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:26.271333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:11:26.272306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:26.273451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 230/439] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:11:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:11:26.282347 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b67f18-6103-415a-9897-3431edd9fecf None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:26.458167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6632e-0d48-4889-9282-2f7dd314ef53 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:26.491746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b67f18-6103-415a-9897-3431edd9fecf None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:26.525794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:26.536978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4aeb7-fcec-4bb8-a847-8f99849fee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:26.538472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4aeb7-fcec-4bb8-a847-8f99849fee6a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:26.539039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4aeb7-fcec-4bb8-a847-8f99849fee6a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:26.539595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e4aeb7-fcec-4bb8-a847-8f99849fee6a None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:26.542133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4aeb7-fcec-4bb8-a847-8f99849fee6a None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:26.545209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e4aeb7-fcec-4bb8-a847-8f99849fee6a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:26.548487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 210/440] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:26 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:26.558068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4aeb7-fcec-4bb8-a847-8f99849fee6a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:26.565608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073c6cf-f4d6-4422-9476-bcfa2c3ae554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:26.566667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073c6cf-f4d6-4422-9476-bcfa2c3ae554 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:26.567181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073c6cf-f4d6-4422-9476-bcfa2c3ae554 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:26.567669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073c6cf-f4d6-4422-9476-bcfa2c3ae554 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:26.568976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073c6cf-f4d6-4422-9476-bcfa2c3ae554 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:26.569360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073c6cf-f4d6-4422-9476-bcfa2c3ae554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:26.572082 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073c6cf-f4d6-4422-9476-bcfa2c3ae554 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:26.595665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 231/441] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:26.599158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073c6cf-f4d6-4422-9476-bcfa2c3ae554 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:26.670056 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:26.784996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e4aeb7-fcec-4bb8-a847-8f99849fee6a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:26.805231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073c6cf-f4d6-4422-9476-bcfa2c3ae554 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:26.845093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e434dd8-31a0-4260-a63f-3fb90e472da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:26.846424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e434dd8-31a0-4260-a63f-3fb90e472da2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:26.846806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e434dd8-31a0-4260-a63f-3fb90e472da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:26.847265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e434dd8-31a0-4260-a63f-3fb90e472da2 None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:26.849426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e434dd8-31a0-4260-a63f-3fb90e472da2 None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:26.853045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e434dd8-31a0-4260-a63f-3fb90e472da2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:26.855689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 211/442] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:26 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:26.857857 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbdec2-1803-43c5-8068-b44ab073f734 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:26.858311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbdec2-1803-43c5-8068-b44ab073f734 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:26.858802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbdec2-1803-43c5-8068-b44ab073f734 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:26.862241 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 212/443] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:26.868306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbdec2-1803-43c5-8068-b44ab073f734 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:26.881439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea9c85-7d0c-4953-bc22-05bc0640056e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:26.882484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea9c85-7d0c-4953-bc22-05bc0640056e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:26.882994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea9c85-7d0c-4953-bc22-05bc0640056e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:26.883435 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea9c85-7d0c-4953-bc22-05bc0640056e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:26.970912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:27.013217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea9c85-7d0c-4953-bc22-05bc0640056e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:27.016008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea9c85-7d0c-4953-bc22-05bc0640056e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:27.059900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbdec2-1803-43c5-8068-b44ab073f734 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:27.089386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:27.123815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e0558-3ea4-4096-8e05-bd2eba82ccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:27.124773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e0558-3ea4-4096-8e05-bd2eba82ccd3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:27.125189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e0558-3ea4-4096-8e05-bd2eba82ccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:27.125626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e0558-3ea4-4096-8e05-bd2eba82ccd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:27.127283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e0558-3ea4-4096-8e05-bd2eba82ccd3 None admin] RBAC: Adding query filter params (group.id=3aa3b4acf6304cc28f9bc849e2ee5593, role.id=ef48ab9c5b844be59351a260130d3071, scope.project.id=366a6c5b1cb444698289e32b2b46918d) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:27.127846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e0558-3ea4-4096-8e05-bd2eba82ccd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:27.130617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e0558-3ea4-4096-8e05-bd2eba82ccd3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:27.140958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 213/444] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Jul 20 08:11:26 2022] GET /identity/v3/role_assignments?group.id=3aa3b4acf6304cc28f9bc849e2ee5593&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=ef48ab9c5b844be59351a260130d3071 => generated 611 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:27.143286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0605e04-dc67-49b7-a819-950e22cf61e3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:27.143805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0605e04-dc67-49b7-a819-950e22cf61e3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:27.144315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0605e04-dc67-49b7-a819-950e22cf61e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:27.183347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d0605e04-dc67-49b7-a819-950e22cf61e3 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:27.185144 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d0605e04-dc67-49b7-a819-950e22cf61e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:27.204429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 232/445] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 08:11:27.213864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea9c85-7d0c-4953-bc22-05bc0640056e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:27.306004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 214/446] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:27.335169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0605e04-dc67-49b7-a819-950e22cf61e3 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:27.372905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:27.422326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea9c85-7d0c-4953-bc22-05bc0640056e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:27.498372 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4ae21-3ab7-4564-8552-0912ab8284e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:27.499862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4ae21-3ab7-4564-8552-0912ab8284e1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:27.500338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4ae21-3ab7-4564-8552-0912ab8284e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:27.500816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4ae21-3ab7-4564-8552-0912ab8284e1 None admin] PATH_INFO: `/v3/projects/10580c2140c64e24a4fb1ff11f27a985` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:27.503108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4ae21-3ab7-4564-8552-0912ab8284e1 None admin] RBAC: Authorizing `identity:get_project(project_id=10580c2140c64e24a4fb1ff11f27a985)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:27.505759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4ae21-3ab7-4564-8552-0912ab8284e1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:27.509273 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 233/447] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:27 2022] GET /identity/v3/projects/10580c2140c64e24a4fb1ff11f27a985 => generated 344 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:27.516684 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4ae21-3ab7-4564-8552-0912ab8284e1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:27.541511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0605e04-dc67-49b7-a819-950e22cf61e3 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:27.565761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:27.622721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ff401-d7e6-4de8-8028-1aeeb6277ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:27.623959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ff401-d7e6-4de8-8028-1aeeb6277ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:27.624435 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ff401-d7e6-4de8-8028-1aeeb6277ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:27.624899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ff401-d7e6-4de8-8028-1aeeb6277ab8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:27.645276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ff401-d7e6-4de8-8028-1aeeb6277ab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-268ff401-d7e6-4de8-8028-1aeeb6277ab8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:27.655406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:27.656360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:27.657540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 215/448] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:11:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:11:27.667739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ff401-d7e6-4de8-8028-1aeeb6277ab8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:27.708789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4ae21-3ab7-4564-8552-0912ab8284e1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:27.752771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe35ff-2c16-4ebf-8680-b6faf618f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:27.753825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe35ff-2c16-4ebf-8680-b6faf618f709 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:27.754276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe35ff-2c16-4ebf-8680-b6faf618f709 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:27.754813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe35ff-2c16-4ebf-8680-b6faf618f709 None admin] PATH_INFO: `/v3/groups/ff0555320df4412e8ea23f2ca97b10d0` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:27.757147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe35ff-2c16-4ebf-8680-b6faf618f709 None admin] RBAC: Authorizing `identity:get_group(group_id=ff0555320df4412e8ea23f2ca97b10d0)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:27.759905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe35ff-2c16-4ebf-8680-b6faf618f709 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:27.762675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 234/449] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:27 2022] GET /identity/v3/groups/ff0555320df4412e8ea23f2ca97b10d0 => generated 229 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:27.769667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe35ff-2c16-4ebf-8680-b6faf618f709 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:27.789712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:27.808462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ff401-d7e6-4de8-8028-1aeeb6277ab8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:27.871278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c32624-b643-4907-b045-bfc789ad749b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:27.873217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c32624-b643-4907-b045-bfc789ad749b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:27.873835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c32624-b643-4907-b045-bfc789ad749b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:27.874817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c32624-b643-4907-b045-bfc789ad749b None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:27.876405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c32624-b643-4907-b045-bfc789ad749b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:27.876974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c32624-b643-4907-b045-bfc789ad749b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:27.879858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c32624-b643-4907-b045-bfc789ad749b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:27.895980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 216/450] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:27.901882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c32624-b643-4907-b045-bfc789ad749b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:27.933598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe35ff-2c16-4ebf-8680-b6faf618f709 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:27.954412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:27.979856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfab4a-c553-4af2-9b00-7fa90bf0fb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:27.980672 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfab4a-c553-4af2-9b00-7fa90bf0fb32 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:27.981124 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfab4a-c553-4af2-9b00-7fa90bf0fb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:27.981479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adfab4a-c553-4af2-9b00-7fa90bf0fb32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:27.982404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfab4a-c553-4af2-9b00-7fa90bf0fb32 None admin] RBAC: Adding query filter params (group.id=ff0555320df4412e8ea23f2ca97b10d0, role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=10580c2140c64e24a4fb1ff11f27a985) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:27.982883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfab4a-c553-4af2-9b00-7fa90bf0fb32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:27.984978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adfab4a-c553-4af2-9b00-7fa90bf0fb32 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:27.993113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 235/451] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Jul 20 08:11:27 2022] GET /identity/v3/role_assignments?group.id=ff0555320df4412e8ea23f2ca97b10d0&scope.project.id=10580c2140c64e24a4fb1ff11f27a985&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 611 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:28.090508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c32624-b643-4907-b045-bfc789ad749b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:28.151862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a9f48-0b32-42fd-9962-5da7d56a712f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:28.152595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a9f48-0b32-42fd-9962-5da7d56a712f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:28.152883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a9f48-0b32-42fd-9962-5da7d56a712f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:28.153156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a9f48-0b32-42fd-9962-5da7d56a712f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:28.283022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a9f48-0b32-42fd-9962-5da7d56a712f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:28.286693 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a9f48-0b32-42fd-9962-5da7d56a712f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:28.333291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:28.469015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 217/452] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 08:11:28.477158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfab4a-c553-4af2-9b00-7fa90bf0fb32 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:28.598837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:28.627772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adfab4a-c553-4af2-9b00-7fa90bf0fb32 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:28.684486 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1ef92-2797-4237-9eb6-dfd38bc228a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:28.686371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1ef92-2797-4237-9eb6-dfd38bc228a8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:28.686939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1ef92-2797-4237-9eb6-dfd38bc228a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:28.687430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1ef92-2797-4237-9eb6-dfd38bc228a8 None admin] PATH_INFO: `/v3/projects/366a6c5b1cb444698289e32b2b46918d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:28.689641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1ef92-2797-4237-9eb6-dfd38bc228a8 None admin] RBAC: Authorizing `identity:get_project(project_id=366a6c5b1cb444698289e32b2b46918d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:28.692096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1ef92-2797-4237-9eb6-dfd38bc228a8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:28.695285 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 236/453] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:11:28 2022] GET /identity/v3/projects/366a6c5b1cb444698289e32b2b46918d => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:28.702858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a9f48-0b32-42fd-9962-5da7d56a712f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:28.838193 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a9f48-0b32-42fd-9962-5da7d56a712f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:28.873160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:28.882823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc3b9a-c3a3-4e8b-93e0-5e95a2a75b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:28.883716 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc3b9a-c3a3-4e8b-93e0-5e95a2a75b92 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:28.884108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc3b9a-c3a3-4e8b-93e0-5e95a2a75b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:28.884570 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc3b9a-c3a3-4e8b-93e0-5e95a2a75b92 None admin] PATH_INFO: `/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:28.886010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc3b9a-c3a3-4e8b-93e0-5e95a2a75b92 None admin] RBAC: Authorizing `identity:get_group(group_id=3aa3b4acf6304cc28f9bc849e2ee5593)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:28.887630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc3b9a-c3a3-4e8b-93e0-5e95a2a75b92 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:28.889715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 218/454] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Jul 20 08:11:28 2022] GET /identity/v3/groups/3aa3b4acf6304cc28f9bc849e2ee5593 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:28.897629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1ef92-2797-4237-9eb6-dfd38bc228a8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:29.038964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1ef92-2797-4237-9eb6-dfd38bc228a8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:29.084072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ef7b9-f6c3-4746-98e7-8be403606879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:29.085003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ef7b9-f6c3-4746-98e7-8be403606879 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:29.085416 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ef7b9-f6c3-4746-98e7-8be403606879 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:29.085882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ef7b9-f6c3-4746-98e7-8be403606879 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:29.087116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ef7b9-f6c3-4746-98e7-8be403606879 None admin] RBAC: Adding query filter params (group.id=3aa3b4acf6304cc28f9bc849e2ee5593, role.id=1d508b4effb546039e87dd273933af16, scope.project.id=366a6c5b1cb444698289e32b2b46918d) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:29.087534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ef7b9-f6c3-4746-98e7-8be403606879 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:29.089255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ef7b9-f6c3-4746-98e7-8be403606879 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:29.097058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 237/455] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Jul 20 08:11:28 2022] GET /identity/v3/role_assignments?group.id=3aa3b4acf6304cc28f9bc849e2ee5593&scope.project.id=366a6c5b1cb444698289e32b2b46918d&role.id=1d508b4effb546039e87dd273933af16 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:33.386844 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239cd72-feb7-43b4-b0f5-ddaf2a7fb955 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:33.388473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239cd72-feb7-43b4-b0f5-ddaf2a7fb955 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:33.389274 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239cd72-feb7-43b4-b0f5-ddaf2a7fb955 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:33.392857 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 219/456] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:33.402051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3042b1b-5700-4b58-8816-302a089fe862 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:33.403687 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3042b1b-5700-4b58-8816-302a089fe862 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:33.404489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3042b1b-5700-4b58-8816-302a089fe862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:33.444117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d3042b1b-5700-4b58-8816-302a089fe862 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:33.446816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d3042b1b-5700-4b58-8816-302a089fe862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:33.453476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:33.608725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 238/457] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:33.686166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239cd72-feb7-43b4-b0f5-ddaf2a7fb955 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:33.718199 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d833f03-4c28-4cdb-9076-3e6e044289ae None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:33.719094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d833f03-4c28-4cdb-9076-3e6e044289ae None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:33.720780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d833f03-4c28-4cdb-9076-3e6e044289ae None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:33.724638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 239/458] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:11:33.734568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b56226-428b-4e35-92d5-4cc750033d9e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:33.735908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b56226-428b-4e35-92d5-4cc750033d9e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:33.737163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b56226-428b-4e35-92d5-4cc750033d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:33.769884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-10b56226-428b-4e35-92d5-4cc750033d9e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:33.772823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-10b56226-428b-4e35-92d5-4cc750033d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:33.971095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 240/459] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 08:11:34.036621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b56226-428b-4e35-92d5-4cc750033d9e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:34.075708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239cd72-feb7-43b4-b0f5-ddaf2a7fb955 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:34.220940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110922ad-c007-4d83-9906-6c15e8ea58c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:34.223013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110922ad-c007-4d83-9906-6c15e8ea58c7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:34.223862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110922ad-c007-4d83-9906-6c15e8ea58c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:34.224642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110922ad-c007-4d83-9906-6c15e8ea58c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:34.233822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:34.264956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110922ad-c007-4d83-9906-6c15e8ea58c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:34.268206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110922ad-c007-4d83-9906-6c15e8ea58c7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:34.296896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b56226-428b-4e35-92d5-4cc750033d9e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-110922ad-c007-4d83-9906-6c15e8ea58c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:34.307315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:34.309159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:34.311129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:34.311129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:34.311129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:34.311129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:34.314513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 220/460] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 629 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:34.317527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454229a-5dab-46cc-88e5-ccfa706859c2 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:34.318380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454229a-5dab-46cc-88e5-ccfa706859c2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:34.319194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454229a-5dab-46cc-88e5-ccfa706859c2 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:34.322665 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 221/461] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:34.332384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d40ad-f439-478f-802d-01c3d4340861 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:34.333266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d40ad-f439-478f-802d-01c3d4340861 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:34.334011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d40ad-f439-478f-802d-01c3d4340861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:34.358310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab5b5a-a4e6-4cb6-a28a-88db8d7f0dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:34.359086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab5b5a-a4e6-4cb6-a28a-88db8d7f0dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:34.361144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab5b5a-a4e6-4cb6-a28a-88db8d7f0dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:34.361144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab5b5a-a4e6-4cb6-a28a-88db8d7f0dd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:34.369066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-151d40ad-f439-478f-802d-01c3d4340861 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:34.371632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-151d40ad-f439-478f-802d-01c3d4340861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:34.379892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab5b5a-a4e6-4cb6-a28a-88db8d7f0dd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:34.381220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aab5b5a-a4e6-4cb6-a28a-88db8d7f0dd3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0aab5b5a-a4e6-4cb6-a28a-88db8d7f0dd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:34.403862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:34.406992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:34.409226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:34.409226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:34.409226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:34.409226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:34.409226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 241/462] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:11:34.409226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab5b5a-a4e6-4cb6-a28a-88db8d7f0dd3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:34.483275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:34.527986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 222/463] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:34.530059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d40ad-f439-478f-802d-01c3d4340861 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:34.590129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab5b5a-a4e6-4cb6-a28a-88db8d7f0dd3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:34.634985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:34.656071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795b84a-3e96-4aa8-a8a3-ded746fe49e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:34.660882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795b84a-3e96-4aa8-a8a3-ded746fe49e4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:34.661359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795b84a-3e96-4aa8-a8a3-ded746fe49e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:34.661895 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e795b84a-3e96-4aa8-a8a3-ded746fe49e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:34.663320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795b84a-3e96-4aa8-a8a3-ded746fe49e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:34.663881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795b84a-3e96-4aa8-a8a3-ded746fe49e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:34.666564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e795b84a-3e96-4aa8-a8a3-ded746fe49e4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:34.688623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 242/464] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:34.690774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795b84a-3e96-4aa8-a8a3-ded746fe49e4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:34.743485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d40ad-f439-478f-802d-01c3d4340861 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:34.809396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3863f-c615-48d6-97f6-f3aed9dd40eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:34.810477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3863f-c615-48d6-97f6-f3aed9dd40eb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:34.811018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3863f-c615-48d6-97f6-f3aed9dd40eb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:34.811432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3863f-c615-48d6-97f6-f3aed9dd40eb None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:34.812718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3863f-c615-48d6-97f6-f3aed9dd40eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:34.813228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3863f-c615-48d6-97f6-f3aed9dd40eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:34.816972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3863f-c615-48d6-97f6-f3aed9dd40eb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:34.840861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 223/465] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:34.843801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3863f-c615-48d6-97f6-f3aed9dd40eb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:34.888190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e795b84a-3e96-4aa8-a8a3-ded746fe49e4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:34.937198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:34.967597 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f36a1-5661-4356-97f2-081d8d624fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:34.969051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f36a1-5661-4356-97f2-081d8d624fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:34.969570 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f36a1-5661-4356-97f2-081d8d624fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:34.970084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f36a1-5661-4356-97f2-081d8d624fe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:34.990243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f36a1-5661-4356-97f2-081d8d624fe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:34.993836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f36a1-5661-4356-97f2-081d8d624fe4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-335f36a1-5661-4356-97f2-081d8d624fe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:35.026793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:35.028142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:35.030470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:35.030470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:35.030470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:35.030470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:35.030470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 243/466] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:35.031659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ecbce-4108-4980-8dd8-e8c1a1d793d2 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:35.032118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ecbce-4108-4980-8dd8-e8c1a1d793d2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:35.032610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ecbce-4108-4980-8dd8-e8c1a1d793d2 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:35.036253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 244/467] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:35.037760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ecbce-4108-4980-8dd8-e8c1a1d793d2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:35.072438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3863f-c615-48d6-97f6-f3aed9dd40eb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:35.105280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:35.145054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bb855-31c4-4035-87f7-397dc78f2fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:35.146255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bb855-31c4-4035-87f7-397dc78f2fa2 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:35.146850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bb855-31c4-4035-87f7-397dc78f2fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:35.147154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bb855-31c4-4035-87f7-397dc78f2fa2 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:35.148399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bb855-31c4-4035-87f7-397dc78f2fa2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:35.151010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bb855-31c4-4035-87f7-397dc78f2fa2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:35.202616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 224/468] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:11:34 2022] POST /identity/v3/users => generated 246 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:11:35.202616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bb855-31c4-4035-87f7-397dc78f2fa2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:35.240954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ecbce-4108-4980-8dd8-e8c1a1d793d2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:35.312855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0578d1-d31b-42b3-a426-ca017f07cff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:35.314375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0578d1-d31b-42b3-a426-ca017f07cff9 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:35.316451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0578d1-d31b-42b3-a426-ca017f07cff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:35.317095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0578d1-d31b-42b3-a426-ca017f07cff9 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:35.318600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0578d1-d31b-42b3-a426-ca017f07cff9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:35.321340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0578d1-d31b-42b3-a426-ca017f07cff9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:35.374705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 245/469] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:11:35 2022] POST /identity/v3/users => generated 248 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:11:35.378848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eecc5-6cfc-4a49-9c2e-2f3d50ee4e01 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:35.379379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eecc5-6cfc-4a49-9c2e-2f3d50ee4e01 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:35.379825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4eecc5-6cfc-4a49-9c2e-2f3d50ee4e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:35.428867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:35.436829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bb855-31c4-4035-87f7-397dc78f2fa2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:35.445331 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-cf4eecc5-6cfc-4a49-9c2e-2f3d50ee4e01 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:35.447389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cf4eecc5-6cfc-4a49-9c2e-2f3d50ee4e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:35.520887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12317a2e-88c2-452e-977b-042b02e5cfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:35.521909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12317a2e-88c2-452e-977b-042b02e5cfe6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:35.522271 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12317a2e-88c2-452e-977b-042b02e5cfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:35.522708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12317a2e-88c2-452e-977b-042b02e5cfe6 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:35.523952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12317a2e-88c2-452e-977b-042b02e5cfe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:35.524410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12317a2e-88c2-452e-977b-042b02e5cfe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:35.526815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12317a2e-88c2-452e-977b-042b02e5cfe6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:35.551300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 225/470] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:35.558887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12317a2e-88c2-452e-977b-042b02e5cfe6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:35.609145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 246/471] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:35.613070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb449cbf-ed7a-4d28-bf4b-90bb01f5f8a3 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:35.613992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb449cbf-ed7a-4d28-bf4b-90bb01f5f8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:35.614824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb449cbf-ed7a-4d28-bf4b-90bb01f5f8a3 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:35.618360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 247/472] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:35.627154 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7218c9-1a35-4051-b084-545f67a7e202 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:35.629669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7218c9-1a35-4051-b084-545f67a7e202 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:35.629901 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7218c9-1a35-4051-b084-545f67a7e202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:35.665496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5e7218c9-1a35-4051-b084-545f67a7e202 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:35.669105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5e7218c9-1a35-4051-b084-545f67a7e202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:35.700779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:35.793812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12317a2e-88c2-452e-977b-042b02e5cfe6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:35.823806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 248/473] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:35.825256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7218c9-1a35-4051-b084-545f67a7e202 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:35.882222 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fd0de-83e3-4842-9408-0eef175ac38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:35.884087 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fd0de-83e3-4842-9408-0eef175ac38c None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:35.884929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fd0de-83e3-4842-9408-0eef175ac38c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:35.885756 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fd0de-83e3-4842-9408-0eef175ac38c None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:35.887603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fd0de-83e3-4842-9408-0eef175ac38c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:35.890629 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01fd0de-83e3-4842-9408-0eef175ac38c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:35.928094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 226/474] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:11:35 2022] POST /identity/v3/users => generated 248 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:11:35.929656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fd0de-83e3-4842-9408-0eef175ac38c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:35.959289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:36.082502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7218c9-1a35-4051-b084-545f67a7e202 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:36.146924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fd0de-83e3-4842-9408-0eef175ac38c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:36.205345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb10db-6117-4595-9aa4-b3f00b7e46f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:36.206852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb10db-6117-4595-9aa4-b3f00b7e46f7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:36.207782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb10db-6117-4595-9aa4-b3f00b7e46f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:36.208528 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fb10db-6117-4595-9aa4-b3f00b7e46f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:36.222615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3311bf-4f30-44cd-b51e-1df64c63527f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:36.224548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3311bf-4f30-44cd-b51e-1df64c63527f None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:36.225410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3311bf-4f30-44cd-b51e-1df64c63527f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:36.226161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3311bf-4f30-44cd-b51e-1df64c63527f None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:36.226809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb10db-6117-4595-9aa4-b3f00b7e46f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:36.228302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3311bf-4f30-44cd-b51e-1df64c63527f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:36.228747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fb10db-6117-4595-9aa4-b3f00b7e46f7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:36.232199 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3311bf-4f30-44cd-b51e-1df64c63527f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:36.245998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.sql.core [None req-8f3311bf-4f30-44cd-b51e-1df64c63527f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 08:11:36.245998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 08:11:36.245998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [parameters: {'id': '64acc6fe2eb2496ca3201452b69c2768', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 08:11:36.245998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80225) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-59fb10db-6117-4595-9aa4-b3f00b7e46f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:36.260714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:36.262480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:36.264756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:36.264756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:36.264756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:36.264756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:36.267008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 249/475] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 443 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:36.271339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb10db-6117-4595-9aa4-b3f00b7e46f7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:36.275124 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-8f3311bf-4f30-44cd-b51e-1df64c63527f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application conn.query(q) Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result.read() Jul 20 08:11:36.291069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 08:11:36.292322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application next(self.gen) Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application yield resource Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application next(self.gen) Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise self.value Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application session.commit() Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 08:11:36.293302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.session.flush() Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application compat.raise_( Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 08:11:36.294124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 08:11:36.295389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application conn.query(q) Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application result.read() Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 08:11:36.296993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [parameters: {'id': '64acc6fe2eb2496ca3201452b69c2768', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:36.304399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 08:11:36.307804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:36.311104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 227/476] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Jul 20 08:11:35 2022] POST /identity/v3/roles => generated 144 bytes in 366 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:36.311104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3311bf-4f30-44cd-b51e-1df64c63527f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:36.475901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:36.508195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fb10db-6117-4595-9aa4-b3f00b7e46f7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:36.512369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3311bf-4f30-44cd-b51e-1df64c63527f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:36.569652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a07771-200e-4fb2-8c46-3fd9c1f26eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:36.571610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a07771-200e-4fb2-8c46-3fd9c1f26eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:36.573078 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a07771-200e-4fb2-8c46-3fd9c1f26eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:36.574318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a07771-200e-4fb2-8c46-3fd9c1f26eb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:36.576069 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a07771-200e-4fb2-8c46-3fd9c1f26eb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:36.577076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a07771-200e-4fb2-8c46-3fd9c1f26eb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:36.581090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a07771-200e-4fb2-8c46-3fd9c1f26eb6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:36.581339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eeeb9f-b635-4e2f-ad43-74d7b07dc2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:36.582525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eeeb9f-b635-4e2f-ad43-74d7b07dc2f7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:36.583125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eeeb9f-b635-4e2f-ad43-74d7b07dc2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:36.583569 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eeeb9f-b635-4e2f-ad43-74d7b07dc2f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:36.600467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 250/477] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:36.604403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eeeb9f-b635-4e2f-ad43-74d7b07dc2f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:36.606396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a07771-200e-4fb2-8c46-3fd9c1f26eb6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d2eeeb9f-b635-4e2f-ad43-74d7b07dc2f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:36.616870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:36.618006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 08:11:36.619036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:36.635791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 228/478] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:11:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:36.644213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eeeb9f-b635-4e2f-ad43-74d7b07dc2f7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:36.798700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:36.856070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eeeb9f-b635-4e2f-ad43-74d7b07dc2f7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:36.944867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a07771-200e-4fb2-8c46-3fd9c1f26eb6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:36.957336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40119058-2ee7-45e7-bf94-c85d692174b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:36.958515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40119058-2ee7-45e7-bf94-c85d692174b2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:36.958996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40119058-2ee7-45e7-bf94-c85d692174b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:36.959435 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40119058-2ee7-45e7-bf94-c85d692174b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:36.960475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40119058-2ee7-45e7-bf94-c85d692174b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:36.960870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40119058-2ee7-45e7-bf94-c85d692174b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:36.962554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40119058-2ee7-45e7-bf94-c85d692174b2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:36.990128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:36.999152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 229/479] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:37.036825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3167856-05bd-4034-bf62-6bbb439914ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:37.038052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3167856-05bd-4034-bf62-6bbb439914ae None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:37.038549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3167856-05bd-4034-bf62-6bbb439914ae None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:37.039096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3167856-05bd-4034-bf62-6bbb439914ae None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:37.040586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3167856-05bd-4034-bf62-6bbb439914ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:37.043487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3167856-05bd-4034-bf62-6bbb439914ae None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:37.103532 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 251/480] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:11:36 2022] POST /identity/v3/users => generated 249 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:11:38.991510 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0709d-8746-4d83-a8cc-bfce10a40ff1 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:38.992379 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0709d-8746-4d83-a8cc-bfce10a40ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:38.992548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0709d-8746-4d83-a8cc-bfce10a40ff1 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:38.995687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 230/481] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:39.007072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e98e1-8844-4727-8f0b-632f7e6978ae None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:39.007072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e98e1-8844-4727-8f0b-632f7e6978ae None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:39.007072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e98e1-8844-4727-8f0b-632f7e6978ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:39.038862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-667e98e1-8844-4727-8f0b-632f7e6978ae None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:39.041080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-667e98e1-8844-4727-8f0b-632f7e6978ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:39.194990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 252/482] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:39.273000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0709d-8746-4d83-a8cc-bfce10a40ff1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:39.284518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace90ab-3921-4fca-9654-c5637ef7744d None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:39.284965 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace90ab-3921-4fca-9654-c5637ef7744d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:39.285428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace90ab-3921-4fca-9654-c5637ef7744d None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:39.287853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 253/483] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:11:39.297257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a136da5-cc84-478c-9c69-c05fbdbf1a62 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:39.298241 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a136da5-cc84-478c-9c69-c05fbdbf1a62 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:39.298414 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a136da5-cc84-478c-9c69-c05fbdbf1a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:39.331485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-1a136da5-cc84-478c-9c69-c05fbdbf1a62 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:39.333907 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1a136da5-cc84-478c-9c69-c05fbdbf1a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:39.479157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:39.495650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 254/484] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:39.545629 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0709d-8746-4d83-a8cc-bfce10a40ff1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:39.557200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a136da5-cc84-478c-9c69-c05fbdbf1a62 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:39.628786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c42363-2346-4181-a94f-1a04dcf3c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:39.630170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c42363-2346-4181-a94f-1a04dcf3c761 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:39.635319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c42363-2346-4181-a94f-1a04dcf3c761 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:39.635965 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c42363-2346-4181-a94f-1a04dcf3c761 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:39.684249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c42363-2346-4181-a94f-1a04dcf3c761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d0c42363-2346-4181-a94f-1a04dcf3c761 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:39.700287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:39.701999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:39.704544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 231/485] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:39.724351 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c42363-2346-4181-a94f-1a04dcf3c761 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:39.803160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a136da5-cc84-478c-9c69-c05fbdbf1a62 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:39.929423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629b34f-105d-4b4a-8ceb-8d343c1556a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:39.930938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629b34f-105d-4b4a-8ceb-8d343c1556a3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:39.931462 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629b34f-105d-4b4a-8ceb-8d343c1556a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:39.931966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629b34f-105d-4b4a-8ceb-8d343c1556a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:39.953949 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629b34f-105d-4b4a-8ceb-8d343c1556a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c629b34f-105d-4b4a-8ceb-8d343c1556a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:39.970720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:39.972429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:39.974677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 255/486] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:39.983976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:39.986626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629b34f-105d-4b4a-8ceb-8d343c1556a3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:40.079387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:40.115309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c42363-2346-4181-a94f-1a04dcf3c761 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:40.219735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d08b55-2746-4ee1-bc62-1a0eb28043d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:40.220966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d08b55-2746-4ee1-bc62-1a0eb28043d0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:40.221422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d08b55-2746-4ee1-bc62-1a0eb28043d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:40.221873 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d08b55-2746-4ee1-bc62-1a0eb28043d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:40.223329 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d08b55-2746-4ee1-bc62-1a0eb28043d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:40.230702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d08b55-2746-4ee1-bc62-1a0eb28043d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:40.232371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d08b55-2746-4ee1-bc62-1a0eb28043d0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:40.259993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 232/487] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:11:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 537 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:40.280028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629b34f-105d-4b4a-8ceb-8d343c1556a3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:40.284786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d08b55-2746-4ee1-bc62-1a0eb28043d0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:40.389191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1db64-33b2-4328-aca7-c4d2ac614308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:40.390317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1db64-33b2-4328-aca7-c4d2ac614308 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:40.390886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1db64-33b2-4328-aca7-c4d2ac614308 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:40.391393 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1db64-33b2-4328-aca7-c4d2ac614308 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:40.393048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1db64-33b2-4328-aca7-c4d2ac614308 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:40.393620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1db64-33b2-4328-aca7-c4d2ac614308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:40.403806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1db64-33b2-4328-aca7-c4d2ac614308 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:40.431621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 256/488] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:11:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 449 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:40.438840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1db64-33b2-4328-aca7-c4d2ac614308 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:40.503058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d08b55-2746-4ee1-bc62-1a0eb28043d0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:40.550976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:40.606224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed663362-540f-4cde-b85d-984fc5dc66b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:40.608031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed663362-540f-4cde-b85d-984fc5dc66b8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:40.608793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed663362-540f-4cde-b85d-984fc5dc66b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:40.609360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed663362-540f-4cde-b85d-984fc5dc66b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:40.630389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed663362-540f-4cde-b85d-984fc5dc66b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:40.633152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed663362-540f-4cde-b85d-984fc5dc66b8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:40.657735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:40.663590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1db64-33b2-4328-aca7-c4d2ac614308 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ed663362-540f-4cde-b85d-984fc5dc66b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:40.664755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:40.666380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:40.668605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:40.668605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:40.668605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:40.668605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:40.670817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 233/489] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:11:40.675535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a58007-d3b4-465e-9318-5aef5a0931fa None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:40.676028 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a58007-d3b4-465e-9318-5aef5a0931fa None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:40.676518 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a58007-d3b4-465e-9318-5aef5a0931fa None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:40.679122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 234/490] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:40.684483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9da7b3-164e-486d-86c3-ffa40f588006 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:40.684483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9da7b3-164e-486d-86c3-ffa40f588006 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:40.684483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9da7b3-164e-486d-86c3-ffa40f588006 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:40.684483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 235/491] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:40.685664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9da7b3-164e-486d-86c3-ffa40f588006 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:40.744291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddedfc94-5b06-4695-87db-f3ab86429724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:40.745725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddedfc94-5b06-4695-87db-f3ab86429724 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:40.746248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddedfc94-5b06-4695-87db-f3ab86429724 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:40.746767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddedfc94-5b06-4695-87db-f3ab86429724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:40.770006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddedfc94-5b06-4695-87db-f3ab86429724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:40.772930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddedfc94-5b06-4695-87db-f3ab86429724 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ddedfc94-5b06-4695-87db-f3ab86429724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:40.807563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:40.809220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:40.810929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:40.810929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:40.810929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:40.810929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:40.810929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 257/492] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:11:40.812206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572334f8-1c10-4438-b094-25113f7555b0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:40.812635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572334f8-1c10-4438-b094-25113f7555b0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:40.813058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572334f8-1c10-4438-b094-25113f7555b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:40.842533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-572334f8-1c10-4438-b094-25113f7555b0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:40.844439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-572334f8-1c10-4438-b094-25113f7555b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:40.879839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9da7b3-164e-486d-86c3-ffa40f588006 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:40.929276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad35c34-78c5-441b-8ac6-988129c10008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:40.930105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad35c34-78c5-441b-8ac6-988129c10008 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:40.930535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad35c34-78c5-441b-8ac6-988129c10008 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:40.931050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad35c34-78c5-441b-8ac6-988129c10008 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:40.931970 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad35c34-78c5-441b-8ac6-988129c10008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:40.932386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad35c34-78c5-441b-8ac6-988129c10008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:40.934228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad35c34-78c5-441b-8ac6-988129c10008 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:40.950747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 236/493] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:40.953205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7100b-0ff9-415f-81b8-e5df84c5d60b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:40.953709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7100b-0ff9-415f-81b8-e5df84c5d60b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:40.957715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7100b-0ff9-415f-81b8-e5df84c5d60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:40.969483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 258/494] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:40.971688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572334f8-1c10-4438-b094-25113f7555b0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:40.988314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c7c7100b-0ff9-415f-81b8-e5df84c5d60b None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:40.989690 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c7c7100b-0ff9-415f-81b8-e5df84c5d60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:41.041893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:41.123942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 237/495] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:41.125435 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7100b-0ff9-415f-81b8-e5df84c5d60b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:41.144576 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:41.187736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572334f8-1c10-4438-b094-25113f7555b0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:41.260551 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6850a1-69c6-42dc-af86-164374dd48a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:41.261321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6850a1-69c6-42dc-af86-164374dd48a1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:41.261609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6850a1-69c6-42dc-af86-164374dd48a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:41.261892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6850a1-69c6-42dc-af86-164374dd48a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:41.262923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6850a1-69c6-42dc-af86-164374dd48a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:41.263241 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6850a1-69c6-42dc-af86-164374dd48a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:41.264984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6850a1-69c6-42dc-af86-164374dd48a1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:41.282361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 259/496] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:41.283249 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6850a1-69c6-42dc-af86-164374dd48a1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:41.321638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7100b-0ff9-415f-81b8-e5df84c5d60b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:41.373561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc11d28-e379-45ac-a85f-3ba9f3b72880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:41.374372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11d28-e379-45ac-a85f-3ba9f3b72880 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:41.374693 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11d28-e379-45ac-a85f-3ba9f3b72880 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:41.374973 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc11d28-e379-45ac-a85f-3ba9f3b72880 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:41.387506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc11d28-e379-45ac-a85f-3ba9f3b72880 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:41.389860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc11d28-e379-45ac-a85f-3ba9f3b72880 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9dc11d28-e379-45ac-a85f-3ba9f3b72880 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:11:41.416556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:41.418004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:41.419427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:41.419427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:41.419427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:11:41.419427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:11:41.419427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:11:41.419427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:11:41.419427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:11:41.419427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:11:41.419427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:11:41.419427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:41.419427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 238/497] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Jul 20 08:11:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:41.420645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc11d28-e379-45ac-a85f-3ba9f3b72880 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:41.465909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:41.475014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6850a1-69c6-42dc-af86-164374dd48a1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:41.542262 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9729ff-5da8-4df5-8317-d07e9e44aaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:41.543748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9729ff-5da8-4df5-8317-d07e9e44aaff None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:41.544256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9729ff-5da8-4df5-8317-d07e9e44aaff None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:41.544755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9729ff-5da8-4df5-8317-d07e9e44aaff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:41.567024 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9729ff-5da8-4df5-8317-d07e9e44aaff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4c9729ff-5da8-4df5-8317-d07e9e44aaff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:41.584240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:41.585418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:41.586337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:41.586986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 260/498] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:41.587995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9729ff-5da8-4df5-8317-d07e9e44aaff None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:41.593674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:41.611396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc11d28-e379-45ac-a85f-3ba9f3b72880 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:41.672741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3cff01-a50e-43c0-a35b-4afb150d7c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:41.674671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cff01-a50e-43c0-a35b-4afb150d7c9f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:41.675139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cff01-a50e-43c0-a35b-4afb150d7c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:41.675836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3cff01-a50e-43c0-a35b-4afb150d7c9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:41.699271 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3cff01-a50e-43c0-a35b-4afb150d7c9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:41.701742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3cff01-a50e-43c0-a35b-4afb150d7c9f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ef3cff01-a50e-43c0-a35b-4afb150d7c9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:41.730464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:41.732405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:41.734416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:41.734416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:41.734416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:41.734416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:41.734416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 239/499] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:41.735611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5f8d3-4ac1-4b1d-9f81-90615628c06a None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:41.735987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5f8d3-4ac1-4b1d-9f81-90615628c06a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:41.736890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5f8d3-4ac1-4b1d-9f81-90615628c06a None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:41.739565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 240/500] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:41.740215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5f8d3-4ac1-4b1d-9f81-90615628c06a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:41.801644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9729ff-5da8-4df5-8317-d07e9e44aaff None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:41.850773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01244fb3-60a6-49ba-a9ef-1b6212391a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:41.852034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01244fb3-60a6-49ba-a9ef-1b6212391a6e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:41.852419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01244fb3-60a6-49ba-a9ef-1b6212391a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:41.852963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01244fb3-60a6-49ba-a9ef-1b6212391a6e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:41.863721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01244fb3-60a6-49ba-a9ef-1b6212391a6e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:41.865354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01244fb3-60a6-49ba-a9ef-1b6212391a6e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:41.871051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-01244fb3-60a6-49ba-a9ef-1b6212391a6e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:11:41.885371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:41.887201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:41.888918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:41.888918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:41.888918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:11:41.888918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:11:41.888918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:11:41.888918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:11:41.888918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:11:41.888918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:11:41.888918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 08:11:41.888918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:41.890816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 261/501] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:41.892698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01244fb3-60a6-49ba-a9ef-1b6212391a6e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:41.901770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5f8d3-4ac1-4b1d-9f81-90615628c06a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:41.944231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6740407-3db1-4f2d-a02c-7b08bb30d963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:41.945120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6740407-3db1-4f2d-a02c-7b08bb30d963 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:41.945410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6740407-3db1-4f2d-a02c-7b08bb30d963 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:41.945689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6740407-3db1-4f2d-a02c-7b08bb30d963 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:41.946569 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6740407-3db1-4f2d-a02c-7b08bb30d963 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:41.946915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6740407-3db1-4f2d-a02c-7b08bb30d963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:41.948411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6740407-3db1-4f2d-a02c-7b08bb30d963 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:41.973181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:41.991746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 241/502] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Jul 20 08:11:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:41.992511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6740407-3db1-4f2d-a02c-7b08bb30d963 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:42.032083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01244fb3-60a6-49ba-a9ef-1b6212391a6e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:42.085345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596106f-ecc2-4915-adee-48d053135363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:42.086189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596106f-ecc2-4915-adee-48d053135363 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:42.086538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596106f-ecc2-4915-adee-48d053135363 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:42.086927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596106f-ecc2-4915-adee-48d053135363 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:42.087860 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596106f-ecc2-4915-adee-48d053135363 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:42.088231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596106f-ecc2-4915-adee-48d053135363 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:42.089820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596106f-ecc2-4915-adee-48d053135363 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:42.101205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 262/503] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:11:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:42.102874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2bc14-2073-40ab-997d-7b7cee3b1245 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:42.103170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2bc14-2073-40ab-997d-7b7cee3b1245 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:42.103551 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2bc14-2073-40ab-997d-7b7cee3b1245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:42.122450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a0a2bc14-2073-40ab-997d-7b7cee3b1245 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:42.123837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a0a2bc14-2073-40ab-997d-7b7cee3b1245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:42.150510 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6740407-3db1-4f2d-a02c-7b08bb30d963 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:42.195782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2948ae3-3b4c-45d5-b0d0-549923843af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:42.196460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2948ae3-3b4c-45d5-b0d0-549923843af9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:42.196748 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2948ae3-3b4c-45d5-b0d0-549923843af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:42.197058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2948ae3-3b4c-45d5-b0d0-549923843af9 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:42.197868 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2948ae3-3b4c-45d5-b0d0-549923843af9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:42.198187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2948ae3-3b4c-45d5-b0d0-549923843af9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:42.199668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2948ae3-3b4c-45d5-b0d0-549923843af9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:42.201728 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:42.224291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 242/504] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:42.225273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2948ae3-3b4c-45d5-b0d0-549923843af9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:42.243133 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 263/505] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:42.244611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2bc14-2073-40ab-997d-7b7cee3b1245 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:42.340546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:42.389580 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2948ae3-3b4c-45d5-b0d0-549923843af9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:42.403265 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2bc14-2073-40ab-997d-7b7cee3b1245 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:42.450286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbeaaf-077e-437d-b77d-31103caaf4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:42.451386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbeaaf-077e-437d-b77d-31103caaf4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:42.451642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758f719-b942-41ab-a7b8-54fb7b34f397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:42.451842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbeaaf-077e-437d-b77d-31103caaf4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:42.452287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbeaaf-077e-437d-b77d-31103caaf4e9 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:42.452657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758f719-b942-41ab-a7b8-54fb7b34f397 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:42.453101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758f719-b942-41ab-a7b8-54fb7b34f397 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:42.453550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758f719-b942-41ab-a7b8-54fb7b34f397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:42.453775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbeaaf-077e-437d-b77d-31103caaf4e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:42.454277 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbeaaf-077e-437d-b77d-31103caaf4e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:42.457410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbeaaf-077e-437d-b77d-31103caaf4e9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:42.490104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 243/506] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Jul 20 08:11:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:42.491496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbeaaf-077e-437d-b77d-31103caaf4e9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:42.572380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758f719-b942-41ab-a7b8-54fb7b34f397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:42.573932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758f719-b942-41ab-a7b8-54fb7b34f397 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:42.586051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:42.640481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:42.649832 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbeaaf-077e-437d-b77d-31103caaf4e9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:42.697351 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b83d72-d632-4bef-8104-34e637167e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:42.698752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b83d72-d632-4bef-8104-34e637167e6c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:42.699254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b83d72-d632-4bef-8104-34e637167e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:42.699575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 264/507] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:42.699780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b83d72-d632-4bef-8104-34e637167e6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:42.700639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758f719-b942-41ab-a7b8-54fb7b34f397 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:42.724755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b83d72-d632-4bef-8104-34e637167e6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:42.727537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b83d72-d632-4bef-8104-34e637167e6c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-44b83d72-d632-4bef-8104-34e637167e6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:42.761824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:42.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:42.764067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:42.764067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:42.764067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:42.764067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:42.764067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 244/508] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:42.765716 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b83d72-d632-4bef-8104-34e637167e6c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:42.862469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758f719-b942-41ab-a7b8-54fb7b34f397 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:42.930259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf89026-0db7-4b65-a5a6-cf8e727ff157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:42.931612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf89026-0db7-4b65-a5a6-cf8e727ff157 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:42.932086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf89026-0db7-4b65-a5a6-cf8e727ff157 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:42.932567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf89026-0db7-4b65-a5a6-cf8e727ff157 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:42.933905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf89026-0db7-4b65-a5a6-cf8e727ff157 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:42.936830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf89026-0db7-4b65-a5a6-cf8e727ff157 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:42.949475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b83d72-d632-4bef-8104-34e637167e6c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:42.952751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:42.972520 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 265/509] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:11:42 2022] POST /identity/v3/users => generated 247 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:11:42.974034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf89026-0db7-4b65-a5a6-cf8e727ff157 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:42.998865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:43.006797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782df9c2-62b2-4720-b439-4ce47996c11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:43.008180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782df9c2-62b2-4720-b439-4ce47996c11f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:43.008664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782df9c2-62b2-4720-b439-4ce47996c11f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:43.009023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782df9c2-62b2-4720-b439-4ce47996c11f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:43.031268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782df9c2-62b2-4720-b439-4ce47996c11f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-782df9c2-62b2-4720-b439-4ce47996c11f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:43.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:43.044610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:43.045848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 245/510] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:43.047179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782df9c2-62b2-4720-b439-4ce47996c11f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:43.166837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf89026-0db7-4b65-a5a6-cf8e727ff157 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:43.216148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bd159-c802-460e-adc2-b70e66dbcaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:43.217181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bd159-c802-460e-adc2-b70e66dbcaba None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:43.217639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bd159-c802-460e-adc2-b70e66dbcaba None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:43.218041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bd159-c802-460e-adc2-b70e66dbcaba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:43.239523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782df9c2-62b2-4720-b439-4ce47996c11f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:43.315774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf6b75-8d92-430d-b9f5-a5081e873c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:43.317145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf6b75-8d92-430d-b9f5-a5081e873c17 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:43.317718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf6b75-8d92-430d-b9f5-a5081e873c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:43.318278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf6b75-8d92-430d-b9f5-a5081e873c17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:43.330226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bd159-c802-460e-adc2-b70e66dbcaba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:43.332579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bd159-c802-460e-adc2-b70e66dbcaba None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:43.340478 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf6b75-8d92-430d-b9f5-a5081e873c17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:43.343305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf6b75-8d92-430d-b9f5-a5081e873c17 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:43.350261 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-edaf6b75-8d92-430d-b9f5-a5081e873c17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:43.381723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:43.383262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:43.385170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:43.385170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:43.385170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:43.385170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:43.385170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 246/511] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:43.385658 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf6b75-8d92-430d-b9f5-a5081e873c17 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:43.422090 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:43.481860 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 266/512] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:43.481860 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bd159-c802-460e-adc2-b70e66dbcaba None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:43.585851 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf6b75-8d92-430d-b9f5-a5081e873c17 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:43.645028 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccfd8d-391d-4785-aaab-4f932addbaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:43.645723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccfd8d-391d-4785-aaab-4f932addbaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:43.646009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccfd8d-391d-4785-aaab-4f932addbaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:43.646290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccfd8d-391d-4785-aaab-4f932addbaa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:43.647275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccfd8d-391d-4785-aaab-4f932addbaa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:43.647590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccfd8d-391d-4785-aaab-4f932addbaa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:43.649790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccfd8d-391d-4785-aaab-4f932addbaa8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:43.658522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bd159-c802-460e-adc2-b70e66dbcaba None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:43.670005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 247/513] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:43.671565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccfd8d-391d-4785-aaab-4f932addbaa8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:43.684164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:43.739828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0f5a3-2100-4d5b-9164-20e83584932c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:43.740880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0f5a3-2100-4d5b-9164-20e83584932c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:43.741312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0f5a3-2100-4d5b-9164-20e83584932c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:43.741735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0f5a3-2100-4d5b-9164-20e83584932c None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:43.743102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0f5a3-2100-4d5b-9164-20e83584932c None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:43.743592 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0f5a3-2100-4d5b-9164-20e83584932c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:43.746074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0f5a3-2100-4d5b-9164-20e83584932c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:43.769580 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 267/514] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:11:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:43.769580 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0f5a3-2100-4d5b-9164-20e83584932c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:43.827866 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:43.868954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccfd8d-391d-4785-aaab-4f932addbaa8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:43.916248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a787c-4ff6-43ac-9909-df224e313c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:43.917127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a787c-4ff6-43ac-9909-df224e313c73 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:43.917551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a787c-4ff6-43ac-9909-df224e313c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:43.917966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a787c-4ff6-43ac-9909-df224e313c73 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:43.918915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a787c-4ff6-43ac-9909-df224e313c73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:43.919409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a787c-4ff6-43ac-9909-df224e313c73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:43.921198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a787c-4ff6-43ac-9909-df224e313c73 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:43.942738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 248/515] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:43.942738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a787c-4ff6-43ac-9909-df224e313c73 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:43.986901 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0f5a3-2100-4d5b-9164-20e83584932c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:44.058907 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af28338-4f9c-40b2-9de4-cd47dacbfda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:44.060355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28338-4f9c-40b2-9de4-cd47dacbfda5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:44.060875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28338-4f9c-40b2-9de4-cd47dacbfda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:44.061420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af28338-4f9c-40b2-9de4-cd47dacbfda5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:44.081460 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af28338-4f9c-40b2-9de4-cd47dacbfda5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:44.083598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af28338-4f9c-40b2-9de4-cd47dacbfda5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-1af28338-4f9c-40b2-9de4-cd47dacbfda5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:44.115087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:44.116957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:44.118897 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:44.118897 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:44.118897 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:44.118897 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:44.124324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 268/516] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:44.125680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af28338-4f9c-40b2-9de4-cd47dacbfda5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:44.131354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a787c-4ff6-43ac-9909-df224e313c73 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:44.172865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:44.199479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b7179-b8ac-4658-9906-eb32c563eb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:44.200657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b7179-b8ac-4658-9906-eb32c563eb97 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:44.201120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b7179-b8ac-4658-9906-eb32c563eb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:44.201578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b7179-b8ac-4658-9906-eb32c563eb97 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:44.207677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:44.217346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b7179-b8ac-4658-9906-eb32c563eb97 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:44.219827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4b7179-b8ac-4658-9906-eb32c563eb97 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-aa4b7179-b8ac-4658-9906-eb32c563eb97 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:11:44.248344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:44.250183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:44.251545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:44.251545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:44.251545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:11:44.251545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:11:44.251545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:11:44.251545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:11:44.251545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:11:44.251545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:11:44.251545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 08:11:44.251545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:44.251545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 249/517] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:44.253296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b7179-b8ac-4658-9906-eb32c563eb97 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:44.309635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af28338-4f9c-40b2-9de4-cd47dacbfda5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:44.370095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae322f8-4d72-40ac-8a91-bb372633f3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:44.371185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae322f8-4d72-40ac-8a91-bb372633f3ad None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:44.371477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae322f8-4d72-40ac-8a91-bb372633f3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:44.371813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae322f8-4d72-40ac-8a91-bb372633f3ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:44.395869 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae322f8-4d72-40ac-8a91-bb372633f3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:44.397743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae322f8-4d72-40ac-8a91-bb372633f3ad None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-aae322f8-4d72-40ac-8a91-bb372633f3ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:44.421221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:44.422885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:44.425925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:44.425925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:44.425925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:44.425925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:44.425925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 269/518] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:44.425925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae322f8-4d72-40ac-8a91-bb372633f3ad None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:44.461911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b7179-b8ac-4658-9906-eb32c563eb97 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:44.509265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631402f-26da-49e3-ac35-28415f11919d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:44.510079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631402f-26da-49e3-ac35-28415f11919d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:44.510374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631402f-26da-49e3-ac35-28415f11919d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:44.510789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631402f-26da-49e3-ac35-28415f11919d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:44.520393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631402f-26da-49e3-ac35-28415f11919d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:44.522127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631402f-26da-49e3-ac35-28415f11919d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-1631402f-26da-49e3-ac35-28415f11919d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:11:44.546923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:44.547826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:44.548738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:11:44.548738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:11:44.548738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:11:44.548738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:11:44.548738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:11:44.548738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:11:44.548738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:44.548738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:44.549497 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 250/519] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:44.550756 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631402f-26da-49e3-ac35-28415f11919d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:44.562061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:44.595908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae322f8-4d72-40ac-8a91-bb372633f3ad None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:44.650683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef174ece-b3da-4bcc-99ae-b6482719b259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:44.651833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef174ece-b3da-4bcc-99ae-b6482719b259 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:44.652344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef174ece-b3da-4bcc-99ae-b6482719b259 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:44.652851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef174ece-b3da-4bcc-99ae-b6482719b259 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:44.654248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef174ece-b3da-4bcc-99ae-b6482719b259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:44.654830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef174ece-b3da-4bcc-99ae-b6482719b259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:44.657534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef174ece-b3da-4bcc-99ae-b6482719b259 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:44.660557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:44.690706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 270/520] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:44.690706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef174ece-b3da-4bcc-99ae-b6482719b259 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:44.728044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631402f-26da-49e3-ac35-28415f11919d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:44.776671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e80edf-bf44-4140-97e7-3bdfcdae364d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:44.779071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e80edf-bf44-4140-97e7-3bdfcdae364d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:44.779567 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e80edf-bf44-4140-97e7-3bdfcdae364d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:44.780064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e80edf-bf44-4140-97e7-3bdfcdae364d None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:44.782300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e80edf-bf44-4140-97e7-3bdfcdae364d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:44.783562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e80edf-bf44-4140-97e7-3bdfcdae364d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:44.785851 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e80edf-bf44-4140-97e7-3bdfcdae364d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:44.824680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 251/521] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Jul 20 08:11:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:44.826841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e80edf-bf44-4140-97e7-3bdfcdae364d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:44.858879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef174ece-b3da-4bcc-99ae-b6482719b259 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:44.925727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d092af-dcae-4fcf-a2c5-cdcdf63700e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:44.926789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d092af-dcae-4fcf-a2c5-cdcdf63700e6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:44.927252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d092af-dcae-4fcf-a2c5-cdcdf63700e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:44.927704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d092af-dcae-4fcf-a2c5-cdcdf63700e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:44.929009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d092af-dcae-4fcf-a2c5-cdcdf63700e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:44.929496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d092af-dcae-4fcf-a2c5-cdcdf63700e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:44.931926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d092af-dcae-4fcf-a2c5-cdcdf63700e6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:44.936133 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:44.953925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 271/522] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:44.956007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d092af-dcae-4fcf-a2c5-cdcdf63700e6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:44.999348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e80edf-bf44-4140-97e7-3bdfcdae364d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:45.038225 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:45.082686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ca86e-1c79-47e5-86d2-2906fda7fb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:45.083765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ca86e-1c79-47e5-86d2-2906fda7fb35 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:45.084279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ca86e-1c79-47e5-86d2-2906fda7fb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:45.084757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ca86e-1c79-47e5-86d2-2906fda7fb35 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:45.086337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ca86e-1c79-47e5-86d2-2906fda7fb35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:45.086923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ca86e-1c79-47e5-86d2-2906fda7fb35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:45.089646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ca86e-1c79-47e5-86d2-2906fda7fb35 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:45.132448 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 252/523] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:11:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:45.134013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ca86e-1c79-47e5-86d2-2906fda7fb35 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:45.175889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d092af-dcae-4fcf-a2c5-cdcdf63700e6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:45.242695 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1bd96-cf29-49dc-871d-a6cd9d5db334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:45.244291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1bd96-cf29-49dc-871d-a6cd9d5db334 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:45.244803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1bd96-cf29-49dc-871d-a6cd9d5db334 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:45.245452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1bd96-cf29-49dc-871d-a6cd9d5db334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:45.260644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1bd96-cf29-49dc-871d-a6cd9d5db334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:45.263458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1bd96-cf29-49dc-871d-a6cd9d5db334 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-f3d1bd96-cf29-49dc-871d-a6cd9d5db334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:11:45.291292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:45.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:45.295436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:11:45.295436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:11:45.295436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:11:45.295436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:11:45.295436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:11:45.295436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:11:45.295436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:45.295436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:45.295436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 272/524] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:45.295436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1bd96-cf29-49dc-871d-a6cd9d5db334 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:45.310886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ca86e-1c79-47e5-86d2-2906fda7fb35 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:45.372635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cd7f0-06d6-41d2-a65e-cb30ba81dd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:45.373725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cd7f0-06d6-41d2-a65e-cb30ba81dd24 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:45.374213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cd7f0-06d6-41d2-a65e-cb30ba81dd24 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:45.374750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cd7f0-06d6-41d2-a65e-cb30ba81dd24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:45.379137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:45.463253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:45.472447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1bd96-cf29-49dc-871d-a6cd9d5db334 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:45.501704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cd7f0-06d6-41d2-a65e-cb30ba81dd24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:45.506484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cd7f0-06d6-41d2-a65e-cb30ba81dd24 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:45.534335 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d8e8c-f521-46fe-b2ae-5ffedeb7af15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:45.535813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d8e8c-f521-46fe-b2ae-5ffedeb7af15 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:45.536099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d8e8c-f521-46fe-b2ae-5ffedeb7af15 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:45.536391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387d8e8c-f521-46fe-b2ae-5ffedeb7af15 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:45.545607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d8e8c-f521-46fe-b2ae-5ffedeb7af15 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:45.547635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387d8e8c-f521-46fe-b2ae-5ffedeb7af15 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-387d8e8c-f521-46fe-b2ae-5ffedeb7af15 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:11:45.569927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:45.571140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:45.572019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:45.572019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:45.572019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:11:45.572019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:11:45.572019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:11:45.572019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:11:45.572019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:11:45.572019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:11:45.572019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 08:11:45.572019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:45.572019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 273/525] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:45 2022] GET /identity/v3/users/neutron => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:45.573500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d8e8c-f521-46fe-b2ae-5ffedeb7af15 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:45.680934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 253/526] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:45.681858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cd7f0-06d6-41d2-a65e-cb30ba81dd24 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:45.737838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387d8e8c-f521-46fe-b2ae-5ffedeb7af15 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:45.766483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:45.796661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83574a7-0b45-45ed-9af3-53600c620f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:45.797461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83574a7-0b45-45ed-9af3-53600c620f53 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:45.797970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83574a7-0b45-45ed-9af3-53600c620f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:45.798484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83574a7-0b45-45ed-9af3-53600c620f53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:45.800645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83574a7-0b45-45ed-9af3-53600c620f53 None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=84f0da8709624834976d0662aeb02ea6) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:45.801204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83574a7-0b45-45ed-9af3-53600c620f53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:45.804074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83574a7-0b45-45ed-9af3-53600c620f53 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:45.816214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 274/527] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:11:45 2022] GET /identity/v3/role_assignments?user.id=84f0da8709624834976d0662aeb02ea6&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:45.818577 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23c742-988d-428a-8a07-d9e3f2fa14f2 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:45.819114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23c742-988d-428a-8a07-d9e3f2fa14f2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:45.819483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23c742-988d-428a-8a07-d9e3f2fa14f2 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:45.821930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 275/528] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:45.823440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23c742-988d-428a-8a07-d9e3f2fa14f2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:45.839696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:45.883951 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cd7f0-06d6-41d2-a65e-cb30ba81dd24 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:45.941929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7865fc-9b3b-4e8f-9760-3e021ae61497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:45.943032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7865fc-9b3b-4e8f-9760-3e021ae61497 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:45.943491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7865fc-9b3b-4e8f-9760-3e021ae61497 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:45.943940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7865fc-9b3b-4e8f-9760-3e021ae61497 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:45.945225 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7865fc-9b3b-4e8f-9760-3e021ae61497 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:45.945517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7865fc-9b3b-4e8f-9760-3e021ae61497 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:45.947889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7865fc-9b3b-4e8f-9760-3e021ae61497 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:45.967598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 254/529] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:11:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:45.969174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7865fc-9b3b-4e8f-9760-3e021ae61497 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:46.051917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23c742-988d-428a-8a07-d9e3f2fa14f2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:46.107532 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcda0e-7d1a-4e6b-9936-c719e3032eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:46.108373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcda0e-7d1a-4e6b-9936-c719e3032eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:46.108656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcda0e-7d1a-4e6b-9936-c719e3032eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:46.108934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcda0e-7d1a-4e6b-9936-c719e3032eb0 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:46.109795 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcda0e-7d1a-4e6b-9936-c719e3032eb0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:46.110421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcda0e-7d1a-4e6b-9936-c719e3032eb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:46.113044 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebcda0e-7d1a-4e6b-9936-c719e3032eb0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:46.161216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 276/530] 10.222.0.64 () {56 vars in 1139 bytes} [Wed Jul 20 08:11:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:46.162763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cf356-906e-43d8-a275-5e606f9bd65a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:46.163055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cf356-906e-43d8-a275-5e606f9bd65a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:46.163478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cf356-906e-43d8-a275-5e606f9bd65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:46.180023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7865fc-9b3b-4e8f-9760-3e021ae61497 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:46.194063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0c8cf356-906e-43d8-a275-5e606f9bd65a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:46.196243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0c8cf356-906e-43d8-a275-5e606f9bd65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:46.254495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edffcc-611e-4f0a-9e5b-9d16ba2c403f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:46.255910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edffcc-611e-4f0a-9e5b-9d16ba2c403f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:46.256405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edffcc-611e-4f0a-9e5b-9d16ba2c403f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:46.256904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59edffcc-611e-4f0a-9e5b-9d16ba2c403f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:46.262124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:46.263292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:46.283275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59edffcc-611e-4f0a-9e5b-9d16ba2c403f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:46.286634 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59edffcc-611e-4f0a-9e5b-9d16ba2c403f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-59edffcc-611e-4f0a-9e5b-9d16ba2c403f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:46.326791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:46.329234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:46.331024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:46.331024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:46.331024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:46.331024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:46.331024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 255/531] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:46.332137 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edffcc-611e-4f0a-9e5b-9d16ba2c403f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:46.348419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 277/532] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:46.350621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cf356-906e-43d8-a275-5e606f9bd65a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:46.551887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cf356-906e-43d8-a275-5e606f9bd65a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:46.553067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59edffcc-611e-4f0a-9e5b-9d16ba2c403f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:46.609989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663366a-6c3f-4ac4-ab00-00670dfa5072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:46.610956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663366a-6c3f-4ac4-ab00-00670dfa5072 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:46.611874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663366a-6c3f-4ac4-ab00-00670dfa5072 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:46.612195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0663366a-6c3f-4ac4-ab00-00670dfa5072 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:46.613225 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663366a-6c3f-4ac4-ab00-00670dfa5072 None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=3778997232d44cd3a3c4fb3d53e4db53) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:46.613544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663366a-6c3f-4ac4-ab00-00670dfa5072 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:46.615118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0663366a-6c3f-4ac4-ab00-00670dfa5072 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:46.621149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecca3c8-6822-418c-8637-e52849db2cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:46.622546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 256/533] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:11:46 2022] GET /identity/v3/role_assignments?user.id=3778997232d44cd3a3c4fb3d53e4db53&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:46.623042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecca3c8-6822-418c-8637-e52849db2cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:46.623457 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663366a-6c3f-4ac4-ab00-00670dfa5072 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:46.624242 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecca3c8-6822-418c-8637-e52849db2cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:46.624984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecca3c8-6822-418c-8637-e52849db2cd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:46.699946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:46.748264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:46.773760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecca3c8-6822-418c-8637-e52849db2cd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:46.775915 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecca3c8-6822-418c-8637-e52849db2cd5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:46.839464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0663366a-6c3f-4ac4-ab00-00670dfa5072 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:46.903929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a6885-49a4-4359-82e2-1e839b137565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:46.905581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a6885-49a4-4359-82e2-1e839b137565 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:46.905975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a6885-49a4-4359-82e2-1e839b137565 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:46.906260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a6885-49a4-4359-82e2-1e839b137565 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:46.907829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a6885-49a4-4359-82e2-1e839b137565 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:46.908982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a6885-49a4-4359-82e2-1e839b137565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:46.911802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a6885-49a4-4359-82e2-1e839b137565 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:46.929793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 278/534] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:46.930667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 257/535] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:46.930840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecca3c8-6822-418c-8637-e52849db2cd5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:46.938839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a6885-49a4-4359-82e2-1e839b137565 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:47.021070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:47.071947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecca3c8-6822-418c-8637-e52849db2cd5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:47.119203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b3927-1ebe-4d3b-9885-be72fbe76207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:47.120129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b3927-1ebe-4d3b-9885-be72fbe76207 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:47.120474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b3927-1ebe-4d3b-9885-be72fbe76207 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:47.120826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b3927-1ebe-4d3b-9885-be72fbe76207 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:47.136803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b3927-1ebe-4d3b-9885-be72fbe76207 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:47.143494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a6885-49a4-4359-82e2-1e839b137565 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-6c7b3927-1ebe-4d3b-9885-be72fbe76207 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:47.146446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:47.148197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 279/536] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:47.149769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b3927-1ebe-4d3b-9885-be72fbe76207 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:47.175963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:47.230246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372371c-6987-484a-8af6-232b50c5610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:47.232312 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372371c-6987-484a-8af6-232b50c5610a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:47.233059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372371c-6987-484a-8af6-232b50c5610a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:47.233901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372371c-6987-484a-8af6-232b50c5610a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:47.249909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372371c-6987-484a-8af6-232b50c5610a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:47.255900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372371c-6987-484a-8af6-232b50c5610a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d372371c-6987-484a-8af6-232b50c5610a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:11:47.287524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:47.289432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:47.291432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:11:47.291432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:11:47.291432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:11:47.291432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:11:47.291432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:11:47.291432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:11:47.291432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:47.291432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:47.292816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b3927-1ebe-4d3b-9885-be72fbe76207 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:47.303133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 258/537] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:47.305161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372371c-6987-484a-8af6-232b50c5610a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:47.331262 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:47.337529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173cf6b-2b00-49d7-9f33-33d1e2910e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:47.338164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173cf6b-2b00-49d7-9f33-33d1e2910e5e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:47.338759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173cf6b-2b00-49d7-9f33-33d1e2910e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:47.339184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e173cf6b-2b00-49d7-9f33-33d1e2910e5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:47.362518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e173cf6b-2b00-49d7-9f33-33d1e2910e5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:47.365769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e173cf6b-2b00-49d7-9f33-33d1e2910e5e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e173cf6b-2b00-49d7-9f33-33d1e2910e5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:47.415548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:47.416618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:47.417653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:47.417653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:47.417653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:47.417653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:47.419510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 280/538] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:47.421834 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173cf6b-2b00-49d7-9f33-33d1e2910e5e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:47.542197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372371c-6987-484a-8af6-232b50c5610a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:47.617401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f812785-beef-4a9f-a542-305fee210900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:47.618747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f812785-beef-4a9f-a542-305fee210900 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:47.619323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f812785-beef-4a9f-a542-305fee210900 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:47.619894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f812785-beef-4a9f-a542-305fee210900 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:47.621551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f812785-beef-4a9f-a542-305fee210900 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:47.622173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f812785-beef-4a9f-a542-305fee210900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:47.625002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f812785-beef-4a9f-a542-305fee210900 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:47.649631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 259/539] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:11:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:47.649631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f812785-beef-4a9f-a542-305fee210900 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:47.660045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e173cf6b-2b00-49d7-9f33-33d1e2910e5e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:47.725143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:47.758720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed965b4-ba7a-4400-8f61-1079eeb664f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:47.760060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed965b4-ba7a-4400-8f61-1079eeb664f4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:47.760517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed965b4-ba7a-4400-8f61-1079eeb664f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:47.760901 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed965b4-ba7a-4400-8f61-1079eeb664f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:47.762421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed965b4-ba7a-4400-8f61-1079eeb664f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:47.763012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed965b4-ba7a-4400-8f61-1079eeb664f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:47.766803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed965b4-ba7a-4400-8f61-1079eeb664f4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:47.804829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 281/540] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:11:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:47.807089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed965b4-ba7a-4400-8f61-1079eeb664f4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:47.895854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f812785-beef-4a9f-a542-305fee210900 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:47.896692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:47.967674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0f286-67b0-4f6c-81ae-7f874baeb122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:47.968803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0f286-67b0-4f6c-81ae-7f874baeb122 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:47.969108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0f286-67b0-4f6c-81ae-7f874baeb122 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:47.969386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0f286-67b0-4f6c-81ae-7f874baeb122 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:47.970313 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0f286-67b0-4f6c-81ae-7f874baeb122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:47.970676 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0f286-67b0-4f6c-81ae-7f874baeb122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:47.973318 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0f286-67b0-4f6c-81ae-7f874baeb122 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:48.000070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 260/541] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:48.001040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed965b4-ba7a-4400-8f61-1079eeb664f4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:48.002763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0f286-67b0-4f6c-81ae-7f874baeb122 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:48.057749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e8c0-2f1b-4140-8bee-737d2a7e01f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:48.059563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e8c0-2f1b-4140-8bee-737d2a7e01f2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:48.060099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e8c0-2f1b-4140-8bee-737d2a7e01f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:48.060579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e8c0-2f1b-4140-8bee-737d2a7e01f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:48.076822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986e8c0-2f1b-4140-8bee-737d2a7e01f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:48.081186 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986e8c0-2f1b-4140-8bee-737d2a7e01f2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-9986e8c0-2f1b-4140-8bee-737d2a7e01f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:48.109209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:48.112148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:48.113386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:48.113386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:48.113386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:48.113386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:48.114839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 282/542] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:48.116568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e8c0-2f1b-4140-8bee-737d2a7e01f2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:48.176335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:48.190725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0f286-67b0-4f6c-81ae-7f874baeb122 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:48.250015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463ef98-90b2-4850-9654-5f36a7d836f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:48.251193 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463ef98-90b2-4850-9654-5f36a7d836f0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:48.252373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463ef98-90b2-4850-9654-5f36a7d836f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:48.252966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463ef98-90b2-4850-9654-5f36a7d836f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:48.254127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463ef98-90b2-4850-9654-5f36a7d836f0 None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=41a647bd72f344fa963dcd3eb5434b6e) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:48.254709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463ef98-90b2-4850-9654-5f36a7d836f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:48.257247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463ef98-90b2-4850-9654-5f36a7d836f0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:48.266918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 261/543] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:11:48 2022] GET /identity/v3/role_assignments?user.id=41a647bd72f344fa963dcd3eb5434b6e&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:48.270459 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463ef98-90b2-4850-9654-5f36a7d836f0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:48.298853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:48.321185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e8c0-2f1b-4140-8bee-737d2a7e01f2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:48.425197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d427ba8-dfc5-4f7c-9f8d-8f0717031dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:48.426800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d427ba8-dfc5-4f7c-9f8d-8f0717031dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:48.427449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d427ba8-dfc5-4f7c-9f8d-8f0717031dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:48.427886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d427ba8-dfc5-4f7c-9f8d-8f0717031dc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:48.430947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463ef98-90b2-4850-9654-5f36a7d836f0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:48.458147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d427ba8-dfc5-4f7c-9f8d-8f0717031dc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:48.461081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d427ba8-dfc5-4f7c-9f8d-8f0717031dc8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:48.480486 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad555-710e-4b18-9604-0e1dcd48889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:48.481851 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad555-710e-4b18-9604-0e1dcd48889d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:48.482473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad555-710e-4b18-9604-0e1dcd48889d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:48.483115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad555-710e-4b18-9604-0e1dcd48889d None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:48.484174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aad555-710e-4b18-9604-0e1dcd48889d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:48.484736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aad555-710e-4b18-9604-0e1dcd48889d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:48.486908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aad555-710e-4b18-9604-0e1dcd48889d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:48.505177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 262/544] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0d427ba8-dfc5-4f7c-9f8d-8f0717031dc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:11:48.510627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:48.511631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:48.512682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:11:48.512682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:11:48.512682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:11:48.512682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:11:48.512682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:11:48.512682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:11:48.512682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:48.512682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:48.512682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 283/545] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:48.513341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad555-710e-4b18-9604-0e1dcd48889d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:48.521030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d427ba8-dfc5-4f7c-9f8d-8f0717031dc8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:48.562129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:48.845685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad555-710e-4b18-9604-0e1dcd48889d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:48.902910 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d427ba8-dfc5-4f7c-9f8d-8f0717031dc8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:48.906427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b50d9-f3dd-4730-84a0-a7aa44e14d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:48.908239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b50d9-f3dd-4730-84a0-a7aa44e14d8d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:48.909004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b50d9-f3dd-4730-84a0-a7aa44e14d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:48.909709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b50d9-f3dd-4730-84a0-a7aa44e14d8d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:48.927470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b50d9-f3dd-4730-84a0-a7aa44e14d8d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:48.931017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b50d9-f3dd-4730-84a0-a7aa44e14d8d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-396b50d9-f3dd-4730-84a0-a7aa44e14d8d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:11:48.957410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:48.959181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:48.960632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:48.960632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:48.960632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:11:48.960632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:11:48.960632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:11:48.960632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:11:48.960632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:11:48.960632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:11:48.960632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:11:48.960632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:48.963907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 263/546] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:11:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:11:48.978350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b50d9-f3dd-4730-84a0-a7aa44e14d8d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:49.048021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcede2-500d-4537-81d2-44c5108b5c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:49.049713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcede2-500d-4537-81d2-44c5108b5c00 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:49.050265 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcede2-500d-4537-81d2-44c5108b5c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:49.050802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcede2-500d-4537-81d2-44c5108b5c00 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:49.052337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcede2-500d-4537-81d2-44c5108b5c00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:49.052870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcede2-500d-4537-81d2-44c5108b5c00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:49.067605 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcede2-500d-4537-81d2-44c5108b5c00 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:49.108373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 284/547] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:11:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 588 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:49.120104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcede2-500d-4537-81d2-44c5108b5c00 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:49.155159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:49.235474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:49.322711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b50d9-f3dd-4730-84a0-a7aa44e14d8d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:49.353732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcede2-500d-4537-81d2-44c5108b5c00 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:49.412183 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48514aee-66c0-40bc-afbf-3155ac1fbb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:49.413313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48514aee-66c0-40bc-afbf-3155ac1fbb9f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:49.413838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48514aee-66c0-40bc-afbf-3155ac1fbb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:49.414354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48514aee-66c0-40bc-afbf-3155ac1fbb9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:49.416314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48514aee-66c0-40bc-afbf-3155ac1fbb9f None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=a3b03383554346138b98becb2828851a) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:49.416991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48514aee-66c0-40bc-afbf-3155ac1fbb9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:49.420188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48514aee-66c0-40bc-afbf-3155ac1fbb9f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:49.433937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 285/548] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:11:49 2022] GET /identity/v3/role_assignments?user.id=a3b03383554346138b98becb2828851a&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:49.445962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f3f4-6376-4970-884a-b6011c9c704a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:49.447324 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f3f4-6376-4970-884a-b6011c9c704a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:49.447866 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f3f4-6376-4970-884a-b6011c9c704a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:49.448398 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f3f4-6376-4970-884a-b6011c9c704a None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:49.450252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f3f4-6376-4970-884a-b6011c9c704a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:49.450896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f3f4-6376-4970-884a-b6011c9c704a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:49.454136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f3f4-6376-4970-884a-b6011c9c704a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:49.505519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 264/549] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Jul 20 08:11:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 528 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:49.515322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48514aee-66c0-40bc-afbf-3155ac1fbb9f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:49.661734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c81e57-6cba-4376-886a-fd3fda3eea71 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:49.663374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c81e57-6cba-4376-886a-fd3fda3eea71 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:49.663903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c81e57-6cba-4376-886a-fd3fda3eea71 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:49.666301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 265/550] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:49.675054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae253b-14df-47c9-9788-c66ca127e2cb None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:49.675940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae253b-14df-47c9-9788-c66ca127e2cb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:49.676592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae253b-14df-47c9-9788-c66ca127e2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:49.731367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-75ae253b-14df-47c9-9788-c66ca127e2cb None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:49.733443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-75ae253b-14df-47c9-9788-c66ca127e2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:49.770323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:49.822848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48514aee-66c0-40bc-afbf-3155ac1fbb9f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:49.936816 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 266/551] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:49.952030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bffa5-0ba1-4fe4-ab17-c67e2851d907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:49.953543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bffa5-0ba1-4fe4-ab17-c67e2851d907 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:49.954000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bffa5-0ba1-4fe4-ab17-c67e2851d907 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:49.954561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bffa5-0ba1-4fe4-ab17-c67e2851d907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:49.983134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae253b-14df-47c9-9788-c66ca127e2cb None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:50.067522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bffa5-0ba1-4fe4-ab17-c67e2851d907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:50.069742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bffa5-0ba1-4fe4-ab17-c67e2851d907 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:50.183302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:50.263118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:50.279997 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae253b-14df-47c9-9788-c66ca127e2cb None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:50.311992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 286/552] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:11:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 798 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:11:50.322523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bffa5-0ba1-4fe4-ab17-c67e2851d907 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:50.356227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb23e2b-7113-4266-adbe-0f3bffab6fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:50.357593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb23e2b-7113-4266-adbe-0f3bffab6fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:50.358048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb23e2b-7113-4266-adbe-0f3bffab6fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:50.358537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb23e2b-7113-4266-adbe-0f3bffab6fc7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:50.399015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb23e2b-7113-4266-adbe-0f3bffab6fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-6fb23e2b-7113-4266-adbe-0f3bffab6fc7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:50.422732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:50.424572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:50.425538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:50.426188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 267/553] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:11:50.445946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb23e2b-7113-4266-adbe-0f3bffab6fc7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:50.580546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bffa5-0ba1-4fe4-ab17-c67e2851d907 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:50.689964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81930812-cd03-466b-a51d-1a7ab74367ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:50.691429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81930812-cd03-466b-a51d-1a7ab74367ed None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:50.692290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81930812-cd03-466b-a51d-1a7ab74367ed None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:50.692750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81930812-cd03-466b-a51d-1a7ab74367ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:50.717779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81930812-cd03-466b-a51d-1a7ab74367ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:50.722098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81930812-cd03-466b-a51d-1a7ab74367ed None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-81930812-cd03-466b-a51d-1a7ab74367ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:50.758706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:50.760387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:50.761733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:50.761733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:50.761733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:50.761733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:50.763767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb23e2b-7113-4266-adbe-0f3bffab6fc7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:50.766126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 287/554] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:50.769097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ee635-fe74-41cf-acdf-987c66d0fa50 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:50.769720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ee635-fe74-41cf-acdf-987c66d0fa50 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:50.770196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ee635-fe74-41cf-acdf-987c66d0fa50 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:50.772336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 288/555] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:50.783784 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ee635-fe74-41cf-acdf-987c66d0fa50 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:50.820508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:50.840596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:50.844359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032d9c9-d305-4d0e-b04d-fb3bcdfde8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:50.845787 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032d9c9-d305-4d0e-b04d-fb3bcdfde8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:50.846306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032d9c9-d305-4d0e-b04d-fb3bcdfde8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:50.846879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032d9c9-d305-4d0e-b04d-fb3bcdfde8d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:50.848437 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032d9c9-d305-4d0e-b04d-fb3bcdfde8d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:50.848977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032d9c9-d305-4d0e-b04d-fb3bcdfde8d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:50.851441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032d9c9-d305-4d0e-b04d-fb3bcdfde8d1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:50.875030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 268/556] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:11:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:50.878385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e812f29-59b0-4ceb-be57-eb46394ab652 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:50.878945 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e812f29-59b0-4ceb-be57-eb46394ab652 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:50.879467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e812f29-59b0-4ceb-be57-eb46394ab652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:50.913785 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3e812f29-59b0-4ceb-be57-eb46394ab652 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:50.915699 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3e812f29-59b0-4ceb-be57-eb46394ab652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:51.039849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ee635-fe74-41cf-acdf-987c66d0fa50 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:51.061215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 269/557] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:51.063592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e812f29-59b0-4ceb-be57-eb46394ab652 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:51.150737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474435e3-c7d3-475f-ab7b-e05d1fba4007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:51.152930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474435e3-c7d3-475f-ab7b-e05d1fba4007 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:51.153657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474435e3-c7d3-475f-ab7b-e05d1fba4007 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:51.154327 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474435e3-c7d3-475f-ab7b-e05d1fba4007 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:51.155893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474435e3-c7d3-475f-ab7b-e05d1fba4007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:51.156635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474435e3-c7d3-475f-ab7b-e05d1fba4007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:51.160242 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474435e3-c7d3-475f-ab7b-e05d1fba4007 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:51.185071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 289/558] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:51.187013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474435e3-c7d3-475f-ab7b-e05d1fba4007 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:51.293064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e812f29-59b0-4ceb-be57-eb46394ab652 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:51.348446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:51.382990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd9829-2318-4764-a4fd-e93e9c6f8761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:51.385286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9829-2318-4764-a4fd-e93e9c6f8761 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:51.386105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9829-2318-4764-a4fd-e93e9c6f8761 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:51.387134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9829-2318-4764-a4fd-e93e9c6f8761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:51.395984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:51.414958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd9829-2318-4764-a4fd-e93e9c6f8761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:51.418831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd9829-2318-4764-a4fd-e93e9c6f8761 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:51.426064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474435e3-c7d3-475f-ab7b-e05d1fba4007 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-3cbd9829-2318-4764-a4fd-e93e9c6f8761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:51.464525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:51.466255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:51.468007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:51.468007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:51.468007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:51.468007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:51.474123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 270/559] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:51.476139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd9829-2318-4764-a4fd-e93e9c6f8761 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:51.512252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b11a1a-0ad8-4f95-9f00-b2f43acceebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:51.514249 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b11a1a-0ad8-4f95-9f00-b2f43acceebd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:51.515057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b11a1a-0ad8-4f95-9f00-b2f43acceebd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:51.515770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b11a1a-0ad8-4f95-9f00-b2f43acceebd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:51.536887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b11a1a-0ad8-4f95-9f00-b2f43acceebd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-72b11a1a-0ad8-4f95-9f00-b2f43acceebd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:51.554081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:51.556277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:51.558321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:51.571114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 290/560] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:51.573139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b11a1a-0ad8-4f95-9f00-b2f43acceebd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:51.699775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd9829-2318-4764-a4fd-e93e9c6f8761 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:51.786302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a2b30-2d6b-48c5-ad1f-042f3854c37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:51.787863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a2b30-2d6b-48c5-ad1f-042f3854c37e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:51.788439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a2b30-2d6b-48c5-ad1f-042f3854c37e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:51.789001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a2b30-2d6b-48c5-ad1f-042f3854c37e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:51.802272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b11a1a-0ad8-4f95-9f00-b2f43acceebd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:51.805719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a2b30-2d6b-48c5-ad1f-042f3854c37e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:51.808926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a2b30-2d6b-48c5-ad1f-042f3854c37e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-526a2b30-2d6b-48c5-ad1f-042f3854c37e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:11:51.846164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:51.848100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:51.849846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:11:51.849846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:11:51.849846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:11:51.849846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:11:51.849846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:11:51.849846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:11:51.849846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:51.849846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:51.855354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 271/561] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:51.857202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a2b30-2d6b-48c5-ad1f-042f3854c37e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:51.884767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f6b76-bcb4-453e-ba65-1eac7ee60d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:51.886386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f6b76-bcb4-453e-ba65-1eac7ee60d80 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:51.887190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f6b76-bcb4-453e-ba65-1eac7ee60d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:51.888071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f6b76-bcb4-453e-ba65-1eac7ee60d80 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:51.889950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f6b76-bcb4-453e-ba65-1eac7ee60d80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:51.890838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f6b76-bcb4-453e-ba65-1eac7ee60d80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:51.894465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f6b76-bcb4-453e-ba65-1eac7ee60d80 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:51.915721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:51.922180 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 291/562] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:51.924477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f6b76-bcb4-453e-ba65-1eac7ee60d80 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:51.947172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:52.076668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a2b30-2d6b-48c5-ad1f-042f3854c37e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:52.112782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f6b76-bcb4-453e-ba65-1eac7ee60d80 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:52.133296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad64da-b61b-42db-a78c-f409f0520a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:52.134444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad64da-b61b-42db-a78c-f409f0520a5a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:52.135007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad64da-b61b-42db-a78c-f409f0520a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:52.135509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad64da-b61b-42db-a78c-f409f0520a5a None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:52.136814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad64da-b61b-42db-a78c-f409f0520a5a None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:52.137206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad64da-b61b-42db-a78c-f409f0520a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:52.139949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad64da-b61b-42db-a78c-f409f0520a5a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:52.161887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 272/563] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:11:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:52.161887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a84b-90e3-4db9-808c-ee95342d4c9c None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:52.161887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a84b-90e3-4db9-808c-ee95342d4c9c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:52.161887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480a84b-90e3-4db9-808c-ee95342d4c9c None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:52.161887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 273/564] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:52.161887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480a84b-90e3-4db9-808c-ee95342d4c9c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:52.188978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794e790-a419-42a4-b88f-d3f914fc3ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:52.189870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e790-a419-42a4-b88f-d3f914fc3ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:52.190315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e790-a419-42a4-b88f-d3f914fc3ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:52.190625 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794e790-a419-42a4-b88f-d3f914fc3ee5 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:52.191449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794e790-a419-42a4-b88f-d3f914fc3ee5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:52.191754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794e790-a419-42a4-b88f-d3f914fc3ee5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:52.193476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794e790-a419-42a4-b88f-d3f914fc3ee5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:52.213508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 292/565] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:11:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:52.213508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794e790-a419-42a4-b88f-d3f914fc3ee5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:52.348232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:52.356269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:52.365997 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480a84b-90e3-4db9-808c-ee95342d4c9c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:52.418595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794e790-a419-42a4-b88f-d3f914fc3ee5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:52.422310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76a0fe-7b33-4247-983c-79f29e74f856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:52.424297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76a0fe-7b33-4247-983c-79f29e74f856 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:52.424819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76a0fe-7b33-4247-983c-79f29e74f856 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:52.425322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76a0fe-7b33-4247-983c-79f29e74f856 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:52.440268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76a0fe-7b33-4247-983c-79f29e74f856 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:52.443081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76a0fe-7b33-4247-983c-79f29e74f856 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:52.463054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c4992-a608-493a-93a7-5fda416ae08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:52.463835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c4992-a608-493a-93a7-5fda416ae08d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:52.464117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c4992-a608-493a-93a7-5fda416ae08d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:52.464390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c4992-a608-493a-93a7-5fda416ae08d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-cc76a0fe-7b33-4247-983c-79f29e74f856 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:11:52.471674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:52.473686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:52.475290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:52.475290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:52.475290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:11:52.475290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:11:52.475290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:11:52.475290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:11:52.475290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:11:52.475290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:11:52.475290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:11:52.475290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:52.475290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 274/566] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Jul 20 08:11:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:52.476632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb98fe-2915-43ec-a9bf-44bcfcf7f174 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:52.476632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb98fe-2915-43ec-a9bf-44bcfcf7f174 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:52.476632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb98fe-2915-43ec-a9bf-44bcfcf7f174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:52.483239 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c4992-a608-493a-93a7-5fda416ae08d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:52.484771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c4992-a608-493a-93a7-5fda416ae08d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:52.500166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ddcb98fe-2915-43ec-a9bf-44bcfcf7f174 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:52.501429 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ddcb98fe-2915-43ec-a9bf-44bcfcf7f174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-7a5c4992-a608-493a-93a7-5fda416ae08d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:52.505998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:52.507815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:52.510326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:52.510326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:52.510326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:52.510326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:52.510326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 293/567] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:52.510326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c4992-a608-493a-93a7-5fda416ae08d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:52.591422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 275/568] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:52.592875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc8c3a-ea67-42f4-9aaf-fbda8c73c877 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:52.593176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc8c3a-ea67-42f4-9aaf-fbda8c73c877 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:52.593471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc8c3a-ea67-42f4-9aaf-fbda8c73c877 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:52.594933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 276/569] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:52.596538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc8c3a-ea67-42f4-9aaf-fbda8c73c877 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:52.659269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c4992-a608-493a-93a7-5fda416ae08d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:52.685037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:52.709878 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544af0f-4ad2-4694-b09f-f0b322ff29fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:52.711020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544af0f-4ad2-4694-b09f-f0b322ff29fc None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:52.711525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544af0f-4ad2-4694-b09f-f0b322ff29fc None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:52.712044 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544af0f-4ad2-4694-b09f-f0b322ff29fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:52.713695 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544af0f-4ad2-4694-b09f-f0b322ff29fc None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=c3cf87f0515b460dad8c0ff5591c2685) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:52.714267 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544af0f-4ad2-4694-b09f-f0b322ff29fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:52.717168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544af0f-4ad2-4694-b09f-f0b322ff29fc None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:52.718885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:52.737259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 294/570] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:11:52 2022] GET /identity/v3/role_assignments?user.id=c3cf87f0515b460dad8c0ff5591c2685&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:52.737259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544af0f-4ad2-4694-b09f-f0b322ff29fc None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:52.759369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc8c3a-ea67-42f4-9aaf-fbda8c73c877 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:52.810932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d3301-60db-4157-b7e6-88dae2ce429c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:52.812110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d3301-60db-4157-b7e6-88dae2ce429c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:52.812574 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d3301-60db-4157-b7e6-88dae2ce429c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:52.813093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d3301-60db-4157-b7e6-88dae2ce429c None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:52.814680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d3301-60db-4157-b7e6-88dae2ce429c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:52.815199 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d3301-60db-4157-b7e6-88dae2ce429c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:52.817847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d3301-60db-4157-b7e6-88dae2ce429c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:52.844886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 277/571] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Jul 20 08:11:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:52.852877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75521fac-c622-406f-95b5-f998940e333f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:52.853224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75521fac-c622-406f-95b5-f998940e333f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:52.853487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75521fac-c622-406f-95b5-f998940e333f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:52.874308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-75521fac-c622-406f-95b5-f998940e333f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:52.875692 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-75521fac-c622-406f-95b5-f998940e333f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:52.899329 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544af0f-4ad2-4694-b09f-f0b322ff29fc None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:52.946633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ff9c0-d9f6-42b4-a378-f78cdf744134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:52.947439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ff9c0-d9f6-42b4-a378-f78cdf744134 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:52.947914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ff9c0-d9f6-42b4-a378-f78cdf744134 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:52.948376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ff9c0-d9f6-42b4-a378-f78cdf744134 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:52.949424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ff9c0-d9f6-42b4-a378-f78cdf744134 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:52.949786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ff9c0-d9f6-42b4-a378-f78cdf744134 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:52.951370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ff9c0-d9f6-42b4-a378-f78cdf744134 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:52.968931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 295/572] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:52.970472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ff9c0-d9f6-42b4-a378-f78cdf744134 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:53.009554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 278/573] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:53.009554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75521fac-c622-406f-95b5-f998940e333f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:53.048500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:53.120929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:53.132220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ff9c0-d9f6-42b4-a378-f78cdf744134 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:53.168599 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75521fac-c622-406f-95b5-f998940e333f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:53.178524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225356c2-964e-40aa-8d28-da8d1a9ce57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:53.179737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225356c2-964e-40aa-8d28-da8d1a9ce57f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:53.180196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225356c2-964e-40aa-8d28-da8d1a9ce57f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:53.180724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225356c2-964e-40aa-8d28-da8d1a9ce57f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:53.194498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225356c2-964e-40aa-8d28-da8d1a9ce57f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-225356c2-964e-40aa-8d28-da8d1a9ce57f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:53.203361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:53.205174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:53.206159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:53.208504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 296/574] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:53.209516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225356c2-964e-40aa-8d28-da8d1a9ce57f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:53.226354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c689c-3f30-4b66-9e97-1df3ff3317f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:53.228100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c689c-3f30-4b66-9e97-1df3ff3317f7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:53.228576 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c689c-3f30-4b66-9e97-1df3ff3317f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:53.228862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c689c-3f30-4b66-9e97-1df3ff3317f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:53.257640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c689c-3f30-4b66-9e97-1df3ff3317f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:53.260541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c689c-3f30-4b66-9e97-1df3ff3317f7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-286c689c-3f30-4b66-9e97-1df3ff3317f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:53.298296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:53.299606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:53.300668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:53.300668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:53.300668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:53.300668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:53.302642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 279/575] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:53.302985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c689c-3f30-4b66-9e97-1df3ff3317f7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:53.378830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225356c2-964e-40aa-8d28-da8d1a9ce57f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:53.382429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:53.441726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbbcc2-c792-459c-a22d-469a55a82b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:53.443050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbbcc2-c792-459c-a22d-469a55a82b47 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:53.443337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbbcc2-c792-459c-a22d-469a55a82b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:53.443621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbbcc2-c792-459c-a22d-469a55a82b47 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:53.454967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbbcc2-c792-459c-a22d-469a55a82b47 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:53.456814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbbcc2-c792-459c-a22d-469a55a82b47 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:53.464030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c689c-3f30-4b66-9e97-1df3ff3317f7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-b5cbbcc2-c792-459c-a22d-469a55a82b47 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:11:53.475799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:53.476930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:53.478259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:53.478259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:53.478259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:11:53.478259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:11:53.478259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:11:53.478259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:11:53.478259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:11:53.478259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:11:53.478259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 08:11:53.478259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:53.480117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 297/576] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:53.481159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbbcc2-c792-459c-a22d-469a55a82b47 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:53.512104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:53.531971 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e69278-d19c-4335-abdb-ce554dc9d604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:53.533481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e69278-d19c-4335-abdb-ce554dc9d604 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:53.533867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e69278-d19c-4335-abdb-ce554dc9d604 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:53.534275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e69278-d19c-4335-abdb-ce554dc9d604 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:53.548918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e69278-d19c-4335-abdb-ce554dc9d604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-45e69278-d19c-4335-abdb-ce554dc9d604 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:53.557263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:53.558258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 280/577] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:53.559466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e69278-d19c-4335-abdb-ce554dc9d604 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:53.663196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbbcc2-c792-459c-a22d-469a55a82b47 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:53.729708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e69278-d19c-4335-abdb-ce554dc9d604 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:53.743203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74712493-f977-46f9-8124-4a42aacc24cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:53.744292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74712493-f977-46f9-8124-4a42aacc24cc None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:53.744750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74712493-f977-46f9-8124-4a42aacc24cc None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:53.745206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74712493-f977-46f9-8124-4a42aacc24cc None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:53.746763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74712493-f977-46f9-8124-4a42aacc24cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:53.746763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74712493-f977-46f9-8124-4a42aacc24cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:53.748920 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74712493-f977-46f9-8124-4a42aacc24cc None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:53.772678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 298/578] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:11:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:53.772678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74712493-f977-46f9-8124-4a42aacc24cc None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:53.793782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3f0d0-f60f-4e25-9ff5-dde0565f3b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:53.794910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3f0d0-f60f-4e25-9ff5-dde0565f3b4b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:53.795371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3f0d0-f60f-4e25-9ff5-dde0565f3b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:53.795849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3f0d0-f60f-4e25-9ff5-dde0565f3b4b None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:53.796951 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3f0d0-f60f-4e25-9ff5-dde0565f3b4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:53.797480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3f0d0-f60f-4e25-9ff5-dde0565f3b4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:53.800609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3f0d0-f60f-4e25-9ff5-dde0565f3b4b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:53.827424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 281/579] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:53.827424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3f0d0-f60f-4e25-9ff5-dde0565f3b4b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:53.832442 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:53.896814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:53.955394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74712493-f977-46f9-8124-4a42aacc24cc None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:53.986653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3f0d0-f60f-4e25-9ff5-dde0565f3b4b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:54.008336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423dc3c-2cfc-4141-960e-160b127e7604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:54.008336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423dc3c-2cfc-4141-960e-160b127e7604 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:54.008336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423dc3c-2cfc-4141-960e-160b127e7604 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:54.008336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423dc3c-2cfc-4141-960e-160b127e7604 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:54.009060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423dc3c-2cfc-4141-960e-160b127e7604 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:54.009060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423dc3c-2cfc-4141-960e-160b127e7604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:54.009060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423dc3c-2cfc-4141-960e-160b127e7604 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:54.035777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 299/580] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Jul 20 08:11:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:54.036410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423dc3c-2cfc-4141-960e-160b127e7604 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:54.059401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534228c-9a31-4cf1-813f-23c6004ef781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:54.060195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534228c-9a31-4cf1-813f-23c6004ef781 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:54.060531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534228c-9a31-4cf1-813f-23c6004ef781 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:54.061337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534228c-9a31-4cf1-813f-23c6004ef781 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:54.061781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534228c-9a31-4cf1-813f-23c6004ef781 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:54.062105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534228c-9a31-4cf1-813f-23c6004ef781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:54.064832 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c534228c-9a31-4cf1-813f-23c6004ef781 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:54.077422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 282/581] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:11:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:54.078201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534228c-9a31-4cf1-813f-23c6004ef781 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:54.180370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:54.205849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423dc3c-2cfc-4141-960e-160b127e7604 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:54.245957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534228c-9a31-4cf1-813f-23c6004ef781 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:54.248373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:54.272539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6b4eb-06c0-47e3-8bbf-9b33ee94e2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:54.274249 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6b4eb-06c0-47e3-8bbf-9b33ee94e2ee None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:54.274838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6b4eb-06c0-47e3-8bbf-9b33ee94e2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:54.275445 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b6b4eb-06c0-47e3-8bbf-9b33ee94e2ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:54.297174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6b4eb-06c0-47e3-8bbf-9b33ee94e2ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:54.300261 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b6b4eb-06c0-47e3-8bbf-9b33ee94e2ee None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:54.304357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6258c8-a6a0-4bbc-a3a9-f23a6f15563c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:54.305372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6258c8-a6a0-4bbc-a3a9-f23a6f15563c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:54.305649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6258c8-a6a0-4bbc-a3a9-f23a6f15563c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:54.305964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6258c8-a6a0-4bbc-a3a9-f23a6f15563c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:54.314494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6258c8-a6a0-4bbc-a3a9-f23a6f15563c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:54.317472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6258c8-a6a0-4bbc-a3a9-f23a6f15563c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-83b6b4eb-06c0-47e3-8bbf-9b33ee94e2ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:54.332070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:54.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:54.336060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:54.336060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:54.336060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:54.336060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-dd6258c8-a6a0-4bbc-a3a9-f23a6f15563c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:11:54.336500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:54.338070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:54.339804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:11:54.339804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:11:54.339804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:11:54.339804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:11:54.339804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:11:54.339804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:11:54.339804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:54.339804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:54.340639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 300/582] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:54.340639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6b4eb-06c0-47e3-8bbf-9b33ee94e2ee None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:54.342754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 283/583] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:54.343488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6258c8-a6a0-4bbc-a3a9-f23a6f15563c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:54.487252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6258c8-a6a0-4bbc-a3a9-f23a6f15563c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:54.546417 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b6b4eb-06c0-47e3-8bbf-9b33ee94e2ee None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:54.555237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac33760-1071-44ec-872f-e6fa8353e370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:54.556644 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac33760-1071-44ec-872f-e6fa8353e370 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:54.557148 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac33760-1071-44ec-872f-e6fa8353e370 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:54.557641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac33760-1071-44ec-872f-e6fa8353e370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:54.579209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac33760-1071-44ec-872f-e6fa8353e370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:54.582733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac33760-1071-44ec-872f-e6fa8353e370 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-5ac33760-1071-44ec-872f-e6fa8353e370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:54.619017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:54.620739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:54.623127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:54.623127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:54.623127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:54.623127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:54.623127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 284/584] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:54.623127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac33760-1071-44ec-872f-e6fa8353e370 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:54.639037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef704c2-96f9-4b31-af13-340ec914c3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:54.640367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef704c2-96f9-4b31-af13-340ec914c3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:54.640858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef704c2-96f9-4b31-af13-340ec914c3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:54.641355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef704c2-96f9-4b31-af13-340ec914c3a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:54.647094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:54.658533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:54.664669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef704c2-96f9-4b31-af13-340ec914c3a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:54.667143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef704c2-96f9-4b31-af13-340ec914c3a4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0ef704c2-96f9-4b31-af13-340ec914c3a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:54.696790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:54.698841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:54.709644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:54.709644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:54.709644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:54.709644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:54.709644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 301/585] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:54.709644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef704c2-96f9-4b31-af13-340ec914c3a4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:54.881724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac33760-1071-44ec-872f-e6fa8353e370 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:54.921792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef704c2-96f9-4b31-af13-340ec914c3a4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:54.945317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e71a9f-5f24-494b-b426-3d17da9eab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:54.947032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e71a9f-5f24-494b-b426-3d17da9eab8a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:54.947032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e71a9f-5f24-494b-b426-3d17da9eab8a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:54.947032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e71a9f-5f24-494b-b426-3d17da9eab8a None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:54.947571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e71a9f-5f24-494b-b426-3d17da9eab8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:54.948020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e71a9f-5f24-494b-b426-3d17da9eab8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:54.950622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e71a9f-5f24-494b-b426-3d17da9eab8a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:54.974837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 285/586] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:54.975382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e71a9f-5f24-494b-b426-3d17da9eab8a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:55.016195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c980ba-0585-469e-b2a6-810dffa6a74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:55.019854 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c980ba-0585-469e-b2a6-810dffa6a74f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:55.020654 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c980ba-0585-469e-b2a6-810dffa6a74f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:55.021261 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c980ba-0585-469e-b2a6-810dffa6a74f None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:55.023034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c980ba-0585-469e-b2a6-810dffa6a74f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:55.023747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c980ba-0585-469e-b2a6-810dffa6a74f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:55.029583 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c980ba-0585-469e-b2a6-810dffa6a74f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:55.057760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 302/587] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:11:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:55.059470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c980ba-0585-469e-b2a6-810dffa6a74f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:55.127055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:55.185744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:55.192527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e71a9f-5f24-494b-b426-3d17da9eab8a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:55.289189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2cdae-1405-4c57-a394-1a1887285f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:55.290733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2cdae-1405-4c57-a394-1a1887285f70 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:55.291427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2cdae-1405-4c57-a394-1a1887285f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:55.292080 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2cdae-1405-4c57-a394-1a1887285f70 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:55.294447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2cdae-1405-4c57-a394-1a1887285f70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:55.295077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2cdae-1405-4c57-a394-1a1887285f70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:55.298156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea2cdae-1405-4c57-a394-1a1887285f70 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:55.307353 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c980ba-0585-469e-b2a6-810dffa6a74f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:55.337469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 286/588] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:55.340237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2cdae-1405-4c57-a394-1a1887285f70 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:55.371804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac74342-a7b9-45e1-b979-ce973382b25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:55.372954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac74342-a7b9-45e1-b979-ce973382b25b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:55.373471 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac74342-a7b9-45e1-b979-ce973382b25b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:55.373958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac74342-a7b9-45e1-b979-ce973382b25b None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:55.375360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac74342-a7b9-45e1-b979-ce973382b25b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:55.375933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac74342-a7b9-45e1-b979-ce973382b25b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:55.379138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac74342-a7b9-45e1-b979-ce973382b25b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:55.414778 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 303/589] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:55.414778 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac74342-a7b9-45e1-b979-ce973382b25b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:55.571966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2cdae-1405-4c57-a394-1a1887285f70 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:55.621517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac74342-a7b9-45e1-b979-ce973382b25b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:55.628353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac8c10-ba87-40fa-a871-33c723fefe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:55.629751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac8c10-ba87-40fa-a871-33c723fefe05 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:55.630258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac8c10-ba87-40fa-a871-33c723fefe05 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:55.630796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac8c10-ba87-40fa-a871-33c723fefe05 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:55.634408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:55.643793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac8c10-ba87-40fa-a871-33c723fefe05 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:55.644896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:55.646219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ac8c10-ba87-40fa-a871-33c723fefe05 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-56ac8c10-ba87-40fa-a871-33c723fefe05 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:11:55.679812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:55.681523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:55.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:55.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:55.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:11:55.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:11:55.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:11:55.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:11:55.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:11:55.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:11:55.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 08:11:55.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:55.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 287/590] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:11:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:55.684687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac8c10-ba87-40fa-a871-33c723fefe05 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:55.700883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac7eb6-1b21-40d3-a521-5537092a46b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:55.702651 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac7eb6-1b21-40d3-a521-5537092a46b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:55.703142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac7eb6-1b21-40d3-a521-5537092a46b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:55.703612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac7eb6-1b21-40d3-a521-5537092a46b7 None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/84f0da8709624834976d0662aeb02ea6/roles/d60298c676854e83b88df984e42773da` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:55.733738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac7eb6-1b21-40d3-a521-5537092a46b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=84f0da8709624834976d0662aeb02ea6, role_id=d60298c676854e83b88df984e42773da)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:55.736951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac7eb6-1b21-40d3-a521-5537092a46b7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:55.758941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 304/591] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:11:55 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/84f0da8709624834976d0662aeb02ea6/roles/d60298c676854e83b88df984e42773da => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:11:55.759947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac7eb6-1b21-40d3-a521-5537092a46b7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:55.900166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac8c10-ba87-40fa-a871-33c723fefe05 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:55.973852 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca8309-f348-4b39-8074-1803872dfe5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:55.974923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac7eb6-1b21-40d3-a521-5537092a46b7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:55.975290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8309-f348-4b39-8074-1803872dfe5a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:55.975791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8309-f348-4b39-8074-1803872dfe5a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:55.976642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8309-f348-4b39-8074-1803872dfe5a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:56.002391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca8309-f348-4b39-8074-1803872dfe5a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:56.006868 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca8309-f348-4b39-8074-1803872dfe5a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-1fca8309-f348-4b39-8074-1803872dfe5a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:11:56.036502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:56.038754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 288/592] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43038d2-22cd-4dac-988c-8d2093daf2c0 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:56.040876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43038d2-22cd-4dac-988c-8d2093daf2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:56.042439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43038d2-22cd-4dac-988c-8d2093daf2c0 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:56.043290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 289/593] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:11:56.044316 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43038d2-22cd-4dac-988c-8d2093daf2c0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:56.073296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a8a7d-929d-4c25-8d8c-50ad69df4255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:56.073296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a8a7d-929d-4c25-8d8c-50ad69df4255 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:56.073296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a8a7d-929d-4c25-8d8c-50ad69df4255 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:56.073296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a8a7d-929d-4c25-8d8c-50ad69df4255 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:56.086902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a8a7d-929d-4c25-8d8c-50ad69df4255 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:56.088853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a8a7d-929d-4c25-8d8c-50ad69df4255 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-825a8a7d-929d-4c25-8d8c-50ad69df4255 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:11:56.113692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:56.115703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:56.117712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:11:56.117712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:11:56.117712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:11:56.117712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:11:56.117712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:11:56.117712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:11:56.117712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:56.117712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:56.118848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 305/594] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:56.120153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a8a7d-929d-4c25-8d8c-50ad69df4255 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:56.159305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:56.178940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:56.275561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43038d2-22cd-4dac-988c-8d2093daf2c0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:56.308394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a8a7d-929d-4c25-8d8c-50ad69df4255 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:56.336438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f5c7a-3692-4e4a-ae4c-ab149c833167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:56.337408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f5c7a-3692-4e4a-ae4c-ab149c833167 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:56.337888 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f5c7a-3692-4e4a-ae4c-ab149c833167 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:56.338445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f5c7a-3692-4e4a-ae4c-ab149c833167 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:56.340044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f5c7a-3692-4e4a-ae4c-ab149c833167 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:56.340677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f5c7a-3692-4e4a-ae4c-ab149c833167 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:56.342628 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f5c7a-3692-4e4a-ae4c-ab149c833167 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:56.375418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0b687-026e-4ca1-a4ef-eed29e6e0955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:56.376776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0b687-026e-4ca1-a4ef-eed29e6e0955 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:56.377138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0b687-026e-4ca1-a4ef-eed29e6e0955 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:56.377637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0b687-026e-4ca1-a4ef-eed29e6e0955 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:56.379288 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0b687-026e-4ca1-a4ef-eed29e6e0955 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:56.379980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0b687-026e-4ca1-a4ef-eed29e6e0955 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:56.380214 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 290/595] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Jul 20 08:11:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:56.381586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dae38-3f4d-4758-bf03-5d9fa260a19d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:56.381894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dae38-3f4d-4758-bf03-5d9fa260a19d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:56.382170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dae38-3f4d-4758-bf03-5d9fa260a19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:56.382772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0b687-026e-4ca1-a4ef-eed29e6e0955 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:56.411053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 306/596] 10.222.0.64 () {56 vars in 1139 bytes} [Wed Jul 20 08:11:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:56.411053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0b687-026e-4ca1-a4ef-eed29e6e0955 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:56.412730 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-528dae38-3f4d-4758-bf03-5d9fa260a19d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:56.414886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-528dae38-3f4d-4758-bf03-5d9fa260a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:56.562626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 291/597] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:56.562626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dae38-3f4d-4758-bf03-5d9fa260a19d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:56.569806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:56.607877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0b687-026e-4ca1-a4ef-eed29e6e0955 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:56.618987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:56.653258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d2590-cd13-4b4b-86aa-de16f57bedf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:56.653925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d2590-cd13-4b4b-86aa-de16f57bedf7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:56.654206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d2590-cd13-4b4b-86aa-de16f57bedf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:56.654482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d2590-cd13-4b4b-86aa-de16f57bedf7 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:56.655325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d2590-cd13-4b4b-86aa-de16f57bedf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:56.655641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d2590-cd13-4b4b-86aa-de16f57bedf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:56.657111 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d2590-cd13-4b4b-86aa-de16f57bedf7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:56.672499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 307/598] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:11:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:56.680289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d2590-cd13-4b4b-86aa-de16f57bedf7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:56.718371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dae38-3f4d-4758-bf03-5d9fa260a19d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:56.775207 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7d7c6-07d8-4053-8544-8fc8c3694257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:56.776104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7d7c6-07d8-4053-8544-8fc8c3694257 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:56.776296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7d7c6-07d8-4053-8544-8fc8c3694257 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:56.776567 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7d7c6-07d8-4053-8544-8fc8c3694257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:56.788193 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7d7c6-07d8-4053-8544-8fc8c3694257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:56.789600 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7d7c6-07d8-4053-8544-8fc8c3694257 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e9c7d7c6-07d8-4053-8544-8fc8c3694257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:56.810768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:56.811798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:56.812680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:56.812680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:56.812680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:56.812680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:56.812680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 292/599] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:56.813096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7d7c6-07d8-4053-8544-8fc8c3694257 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:56.856898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d2590-cd13-4b4b-86aa-de16f57bedf7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:56.924762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4251f-b2f0-4d14-a802-cff19c9550ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:56.925448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4251f-b2f0-4d14-a802-cff19c9550ce None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:56.925731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4251f-b2f0-4d14-a802-cff19c9550ce None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:56.926014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4251f-b2f0-4d14-a802-cff19c9550ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:56.929470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:56.949853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4251f-b2f0-4d14-a802-cff19c9550ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:56.952848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4251f-b2f0-4d14-a802-cff19c9550ce None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:56.977242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:56.987577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7d7c6-07d8-4053-8544-8fc8c3694257 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ede4251f-b2f0-4d14-a802-cff19c9550ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:56.989811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:56.990943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:56.992092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:56.992092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:56.992092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:56.992092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:56.992642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 308/600] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:56.994128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4251f-b2f0-4d14-a802-cff19c9550ce None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:57.056856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30bd4d-26df-40f3-90e9-13570ffa8e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:57.058009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30bd4d-26df-40f3-90e9-13570ffa8e18 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:57.058597 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30bd4d-26df-40f3-90e9-13570ffa8e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:57.059145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30bd4d-26df-40f3-90e9-13570ffa8e18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:57.078719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc30bd4d-26df-40f3-90e9-13570ffa8e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-fc30bd4d-26df-40f3-90e9-13570ffa8e18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:57.093221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:57.094387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:57.096414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:57.102679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 293/601] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:57.102679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30bd4d-26df-40f3-90e9-13570ffa8e18 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:57.187859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4251f-b2f0-4d14-a802-cff19c9550ce None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:57.243537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918b518-d657-4a95-9b4f-01edf33b7a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:57.245317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918b518-d657-4a95-9b4f-01edf33b7a71 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:57.245822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918b518-d657-4a95-9b4f-01edf33b7a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:57.246309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918b518-d657-4a95-9b4f-01edf33b7a71 None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/3778997232d44cd3a3c4fb3d53e4db53/roles/d60298c676854e83b88df984e42773da` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:57.260730 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30bd4d-26df-40f3-90e9-13570ffa8e18 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:57.275536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918b518-d657-4a95-9b4f-01edf33b7a71 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=3778997232d44cd3a3c4fb3d53e4db53, role_id=d60298c676854e83b88df984e42773da)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:57.279943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918b518-d657-4a95-9b4f-01edf33b7a71 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:57.305413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 309/602] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:11:56 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/3778997232d44cd3a3c4fb3d53e4db53/roles/d60298c676854e83b88df984e42773da => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:11:57.306479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918b518-d657-4a95-9b4f-01edf33b7a71 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:57.349573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657be03-97ba-4026-8da7-27545ed896b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:57.350123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657be03-97ba-4026-8da7-27545ed896b8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:57.350564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657be03-97ba-4026-8da7-27545ed896b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:57.350953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657be03-97ba-4026-8da7-27545ed896b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:57.351785 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657be03-97ba-4026-8da7-27545ed896b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:57.352248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657be03-97ba-4026-8da7-27545ed896b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:57.354469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657be03-97ba-4026-8da7-27545ed896b8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:57.376015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 294/603] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:57.376995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657be03-97ba-4026-8da7-27545ed896b8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:57.411089 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:57.419052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:57.509639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918b518-d657-4a95-9b4f-01edf33b7a71 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:57.561423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48bcf2-1053-40d2-8f8d-5282338b1be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:57.562132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48bcf2-1053-40d2-8f8d-5282338b1be3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:57.562415 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48bcf2-1053-40d2-8f8d-5282338b1be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:57.562735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48bcf2-1053-40d2-8f8d-5282338b1be3 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:57.563700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48bcf2-1053-40d2-8f8d-5282338b1be3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:57.564479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48bcf2-1053-40d2-8f8d-5282338b1be3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:57.566642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48bcf2-1053-40d2-8f8d-5282338b1be3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:57.592736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 310/604] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:57.594316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48bcf2-1053-40d2-8f8d-5282338b1be3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:57.610150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657be03-97ba-4026-8da7-27545ed896b8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:57.676311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6590c11-c658-422a-a422-928ccedc2229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:57.677154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6590c11-c658-422a-a422-928ccedc2229 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:57.677556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6590c11-c658-422a-a422-928ccedc2229 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:57.677847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6590c11-c658-422a-a422-928ccedc2229 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:57.678662 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6590c11-c658-422a-a422-928ccedc2229 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:57.679098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6590c11-c658-422a-a422-928ccedc2229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:57.680636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6590c11-c658-422a-a422-928ccedc2229 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:57.692535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 295/605] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:11:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:57.693515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6590c11-c658-422a-a422-928ccedc2229 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:57.771299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:57.779404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48bcf2-1053-40d2-8f8d-5282338b1be3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:57.817579 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:57.851994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6590c11-c658-422a-a422-928ccedc2229 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:57.855444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b409fa7-b0c2-47f0-b4aa-1636f7503653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:57.857091 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b409fa7-b0c2-47f0-b4aa-1636f7503653 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:57.857615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b409fa7-b0c2-47f0-b4aa-1636f7503653 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:57.858236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b409fa7-b0c2-47f0-b4aa-1636f7503653 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:57.872556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b409fa7-b0c2-47f0-b4aa-1636f7503653 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:57.875888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b409fa7-b0c2-47f0-b4aa-1636f7503653 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:57.899230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029581ca-c350-4221-bf1a-a8594f705d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:57.900289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029581ca-c350-4221-bf1a-a8594f705d27 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:57.900583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029581ca-c350-4221-bf1a-a8594f705d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:57.900914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029581ca-c350-4221-bf1a-a8594f705d27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-3b409fa7-b0c2-47f0-b4aa-1636f7503653 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:11:57.905747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:57.907833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:57.911052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:11:57.911052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:11:57.911052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:11:57.911052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:11:57.911052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:11:57.911052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:11:57.911052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:57.911052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:57.911052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 311/606] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:57.911052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b409fa7-b0c2-47f0-b4aa-1636f7503653 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:57.922706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029581ca-c350-4221-bf1a-a8594f705d27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:57.922706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029581ca-c350-4221-bf1a-a8594f705d27 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-029581ca-c350-4221-bf1a-a8594f705d27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:11:57.942200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:57.944585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:57.947249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:11:57.947249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:11:57.947249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:11:57.947249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:11:57.947249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:11:57.947249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:11:57.947249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:11:57.947249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:57.955086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 296/607] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:11:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:57.955086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029581ca-c350-4221-bf1a-a8594f705d27 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:58.157362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b409fa7-b0c2-47f0-b4aa-1636f7503653 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:58.198414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029581ca-c350-4221-bf1a-a8594f705d27 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:58.240496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bde0f-4b86-4e33-a4de-cd935c7c1bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:58.243539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bde0f-4b86-4e33-a4de-cd935c7c1bbf None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:58.250643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bde0f-4b86-4e33-a4de-cd935c7c1bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:58.250643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844bde0f-4b86-4e33-a4de-cd935c7c1bbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:58.266012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d53ffa-0728-49d4-a5e9-2631b09d9013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:58.267413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d53ffa-0728-49d4-a5e9-2631b09d9013 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:58.267716 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d53ffa-0728-49d4-a5e9-2631b09d9013 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:58.268002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d53ffa-0728-49d4-a5e9-2631b09d9013 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:58.268854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d53ffa-0728-49d4-a5e9-2631b09d9013 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:58.269177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d53ffa-0728-49d4-a5e9-2631b09d9013 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:58.271239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d53ffa-0728-49d4-a5e9-2631b09d9013 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:58.275267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:58.277023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bde0f-4b86-4e33-a4de-cd935c7c1bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:58.280693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844bde0f-4b86-4e33-a4de-cd935c7c1bbf None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:58.290345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 297/608] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:11:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:58.291322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d53ffa-0728-49d4-a5e9-2631b09d9013 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-844bde0f-4b86-4e33-a4de-cd935c7c1bbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:58.326334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:58.328137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:58.331601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:58.331601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:58.331601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:58.331601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:58.335249 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 312/609] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:58.337332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bde0f-4b86-4e33-a4de-cd935c7c1bbf None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:58.356658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:58.421948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d53ffa-0728-49d4-a5e9-2631b09d9013 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:58.463159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa24fa-8649-4880-a6a2-660a76f21ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:58.463898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa24fa-8649-4880-a6a2-660a76f21ffc None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:58.464217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa24fa-8649-4880-a6a2-660a76f21ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:58.464494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa24fa-8649-4880-a6a2-660a76f21ffc None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:58.465341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa24fa-8649-4880-a6a2-660a76f21ffc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:58.465764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa24fa-8649-4880-a6a2-660a76f21ffc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:58.467290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa24fa-8649-4880-a6a2-660a76f21ffc None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:58.481507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844bde0f-4b86-4e33-a4de-cd935c7c1bbf None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:58.482331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 298/610] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:11:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:58.483253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa24fa-8649-4880-a6a2-660a76f21ffc None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:58.530395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05658fcb-85b4-4c0d-8217-a9fbbacb63d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:58.531455 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05658fcb-85b4-4c0d-8217-a9fbbacb63d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:58.531973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05658fcb-85b4-4c0d-8217-a9fbbacb63d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:58.532255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05658fcb-85b4-4c0d-8217-a9fbbacb63d0 None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/41a647bd72f344fa963dcd3eb5434b6e/roles/d60298c676854e83b88df984e42773da` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:58.551953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:58.553123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05658fcb-85b4-4c0d-8217-a9fbbacb63d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=41a647bd72f344fa963dcd3eb5434b6e, role_id=d60298c676854e83b88df984e42773da)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:58.555157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05658fcb-85b4-4c0d-8217-a9fbbacb63d0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:58.576492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 313/611] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:11:58 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/41a647bd72f344fa963dcd3eb5434b6e/roles/d60298c676854e83b88df984e42773da => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:11:58.577777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05658fcb-85b4-4c0d-8217-a9fbbacb63d0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:58.634006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa24fa-8649-4880-a6a2-660a76f21ffc None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:58.660999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:58.691269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238be0-3bb9-4e88-90ee-dc45bcc3dc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:58.691977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238be0-3bb9-4e88-90ee-dc45bcc3dc01 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:58.692621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238be0-3bb9-4e88-90ee-dc45bcc3dc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:58.692817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90238be0-3bb9-4e88-90ee-dc45bcc3dc01 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:58.693623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238be0-3bb9-4e88-90ee-dc45bcc3dc01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:58.694090 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238be0-3bb9-4e88-90ee-dc45bcc3dc01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:58.695732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90238be0-3bb9-4e88-90ee-dc45bcc3dc01 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:58.717119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 299/612] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:11:58.717119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238be0-3bb9-4e88-90ee-dc45bcc3dc01 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:58.747398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05658fcb-85b4-4c0d-8217-a9fbbacb63d0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:58.853776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90238be0-3bb9-4e88-90ee-dc45bcc3dc01 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:58.871586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0710b0-bca1-4bc9-ad89-a356487c88af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:58.872581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0710b0-bca1-4bc9-ad89-a356487c88af None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:58.872871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0710b0-bca1-4bc9-ad89-a356487c88af None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:58.873151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0710b0-bca1-4bc9-ad89-a356487c88af None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/a3b03383554346138b98becb2828851a/roles/d60298c676854e83b88df984e42773da` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:58.886986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:58.908910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13cb42-2b6e-46fe-ba14-597ad02c02cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:58.910040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13cb42-2b6e-46fe-ba14-597ad02c02cf None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:58.910437 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13cb42-2b6e-46fe-ba14-597ad02c02cf None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:58.910883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13cb42-2b6e-46fe-ba14-597ad02c02cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:58.921492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0710b0-bca1-4bc9-ad89-a356487c88af None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=a3b03383554346138b98becb2828851a, role_id=d60298c676854e83b88df984e42773da)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:58.923127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0710b0-bca1-4bc9-ad89-a356487c88af None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:58.931236 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13cb42-2b6e-46fe-ba14-597ad02c02cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:58.934420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13cb42-2b6e-46fe-ba14-597ad02c02cf None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:58.943998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 314/613] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:11:58 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/a3b03383554346138b98becb2828851a/roles/d60298c676854e83b88df984e42773da => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ba13cb42-2b6e-46fe-ba14-597ad02c02cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:11:58.966126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:58.967317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:58.968253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:58.968253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:58.968253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:11:58.968253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:11:58.968253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:11:58.968253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:11:58.968253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:11:58.968253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:11:58.968253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:11:58.968253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:58.968807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 300/614] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:11:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:58.975307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0710b0-bca1-4bc9-ad89-a356487c88af None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:59.037328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcea1f-278d-4bd7-a0c9-88282fd1c1de None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:59.038120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcea1f-278d-4bd7-a0c9-88282fd1c1de None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:59.038647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcea1f-278d-4bd7-a0c9-88282fd1c1de None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:59.041778 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 301/615] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:11:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:11:59.051532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24170e3a-e476-476a-93cf-18344e5fc181 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:59.052036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24170e3a-e476-476a-93cf-18344e5fc181 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:59.052398 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24170e3a-e476-476a-93cf-18344e5fc181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:59.105561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-24170e3a-e476-476a-93cf-18344e5fc181 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:11:59.112456 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-24170e3a-e476-476a-93cf-18344e5fc181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:11:59.258974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 302/616] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:11:59.261537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0710b0-bca1-4bc9-ad89-a356487c88af None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:59.286811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:59.308820 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24170e3a-e476-476a-93cf-18344e5fc181 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:59.352850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef283c-9114-46ac-9f7a-d02b5746f70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:59.353661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef283c-9114-46ac-9f7a-d02b5746f70c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:59.353950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef283c-9114-46ac-9f7a-d02b5746f70c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:59.354226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef283c-9114-46ac-9f7a-d02b5746f70c None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:59.355135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef283c-9114-46ac-9f7a-d02b5746f70c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:11:59.355447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef283c-9114-46ac-9f7a-d02b5746f70c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:59.356994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef283c-9114-46ac-9f7a-d02b5746f70c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:59.390732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 315/617] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Jul 20 08:11:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:11:59.397132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef283c-9114-46ac-9f7a-d02b5746f70c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:59.543920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24170e3a-e476-476a-93cf-18344e5fc181 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:59.558602 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:59.609432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef283c-9114-46ac-9f7a-d02b5746f70c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:11:59.645132 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349a710-3ba8-4c01-a3fc-328c51a60a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:59.646143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349a710-3ba8-4c01-a3fc-328c51a60a03 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:59.646439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349a710-3ba8-4c01-a3fc-328c51a60a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:59.646790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349a710-3ba8-4c01-a3fc-328c51a60a03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:59.672725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349a710-3ba8-4c01-a3fc-328c51a60a03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9349a710-3ba8-4c01-a3fc-328c51a60a03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:11:59.683590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:59.687152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:11:59.689652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:11:59.691307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65a259-5bd4-4f13-8224-7e0b70b927c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:11:59.691307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65a259-5bd4-4f13-8224-7e0b70b927c0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:11:59.691608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 303/618] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:11:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:11:59.691749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65a259-5bd4-4f13-8224-7e0b70b927c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:11:59.691749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65a259-5bd4-4f13-8224-7e0b70b927c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:11:59.700447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349a710-3ba8-4c01-a3fc-328c51a60a03 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:59.726619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65a259-5bd4-4f13-8224-7e0b70b927c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:11:59.726619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65a259-5bd4-4f13-8224-7e0b70b927c0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-bd65a259-5bd4-4f13-8224-7e0b70b927c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:11:59.749298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:11:59.751105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:11:59.753552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:11:59.753552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:11:59.753552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:11:59.753552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:11:59.753552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 316/619] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:11:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:11:59.762096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65a259-5bd4-4f13-8224-7e0b70b927c0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:11:59.794224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:11:59.897899 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65a259-5bd4-4f13-8224-7e0b70b927c0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:00.034626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bff27-9675-4a57-87f3-0a6812fb40ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:00.034626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bff27-9675-4a57-87f3-0a6812fb40ac None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:00.034626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bff27-9675-4a57-87f3-0a6812fb40ac None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:00.034626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bff27-9675-4a57-87f3-0a6812fb40ac None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:00.035250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bff27-9675-4a57-87f3-0a6812fb40ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:00.035250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bff27-9675-4a57-87f3-0a6812fb40ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:00.037779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bff27-9675-4a57-87f3-0a6812fb40ac None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:00.059610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 317/620] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:00.071671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bff27-9675-4a57-87f3-0a6812fb40ac None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:00.083331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349a710-3ba8-4c01-a3fc-328c51a60a03 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:00.152613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea5612-9af2-4de5-b9d7-a44f898ed79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:00.153213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea5612-9af2-4de5-b9d7-a44f898ed79f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:00.153499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea5612-9af2-4de5-b9d7-a44f898ed79f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:00.153778 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea5612-9af2-4de5-b9d7-a44f898ed79f None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:00.154561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea5612-9af2-4de5-b9d7-a44f898ed79f None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:00.154899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea5612-9af2-4de5-b9d7-a44f898ed79f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:00.156342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea5612-9af2-4de5-b9d7-a44f898ed79f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:00.179567 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 304/621] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:11:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:00.191270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea5612-9af2-4de5-b9d7-a44f898ed79f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:00.232680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:00.241802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:00.263276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bff27-9675-4a57-87f3-0a6812fb40ac None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:00.317494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae94d30-4b3f-485e-9408-d45581465dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:00.318379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae94d30-4b3f-485e-9408-d45581465dbe None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:00.319113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae94d30-4b3f-485e-9408-d45581465dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:00.319853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae94d30-4b3f-485e-9408-d45581465dbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:00.338506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae94d30-4b3f-485e-9408-d45581465dbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:00.340749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae94d30-4b3f-485e-9408-d45581465dbe None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-6ae94d30-4b3f-485e-9408-d45581465dbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:00.372859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:00.374074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:00.374957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:00.374957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:00.374957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:00.374957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:00.374957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:00.374957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:00.374957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:00.374957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:00.381356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 318/622] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:00.382763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae94d30-4b3f-485e-9408-d45581465dbe None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:00.497579 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea5612-9af2-4de5-b9d7-a44f898ed79f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:00.511898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae94d30-4b3f-485e-9408-d45581465dbe None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:00.597182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d922ba-9dfa-4376-8e1e-889257631788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:00.597535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973f63e-8972-4e1e-b0dc-d25d5a4e7927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:00.598449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d922ba-9dfa-4376-8e1e-889257631788 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:00.598753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973f63e-8972-4e1e-b0dc-d25d5a4e7927 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:00.599210 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d922ba-9dfa-4376-8e1e-889257631788 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:00.599315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973f63e-8972-4e1e-b0dc-d25d5a4e7927 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:00.599755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973f63e-8972-4e1e-b0dc-d25d5a4e7927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:00.599981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d922ba-9dfa-4376-8e1e-889257631788 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:00.601004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d922ba-9dfa-4376-8e1e-889257631788 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:00.601521 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d922ba-9dfa-4376-8e1e-889257631788 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:00.610773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d922ba-9dfa-4376-8e1e-889257631788 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:00.635058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:00.638475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973f63e-8972-4e1e-b0dc-d25d5a4e7927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:00.640580 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973f63e-8972-4e1e-b0dc-d25d5a4e7927 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:00.650744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 319/623] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:00.653573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080e0c-0a70-4373-8c08-61968fbf7ef4 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:00.654162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080e0c-0a70-4373-8c08-61968fbf7ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:00.654706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41080e0c-0a70-4373-8c08-61968fbf7ef4 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:00.658293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 320/624] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:00.659676 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41080e0c-0a70-4373-8c08-61968fbf7ef4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e973f63e-8972-4e1e-b0dc-d25d5a4e7927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:00.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:00.675929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:00.676797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:00.676797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:00.676797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:00.676797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:00.683267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 305/625] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:00.686012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722fa52-c6ca-4d06-add8-c5a1bbafc896 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:00.686623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722fa52-c6ca-4d06-add8-c5a1bbafc896 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:00.687106 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6722fa52-c6ca-4d06-add8-c5a1bbafc896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:00.719494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6722fa52-c6ca-4d06-add8-c5a1bbafc896 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:00.723591 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6722fa52-c6ca-4d06-add8-c5a1bbafc896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:00.826981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41080e0c-0a70-4373-8c08-61968fbf7ef4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:00.878984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 306/626] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:00.879999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722fa52-c6ca-4d06-add8-c5a1bbafc896 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:00.893572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342e49c-217d-46ee-abb1-fd0d5863c14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:00.895622 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342e49c-217d-46ee-abb1-fd0d5863c14a None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:00.896434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342e49c-217d-46ee-abb1-fd0d5863c14a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:00.897123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3342e49c-217d-46ee-abb1-fd0d5863c14a None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/c3cf87f0515b460dad8c0ff5591c2685/roles/d60298c676854e83b88df984e42773da` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:00.909337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:00.935847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342e49c-217d-46ee-abb1-fd0d5863c14a None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=c3cf87f0515b460dad8c0ff5591c2685, role_id=d60298c676854e83b88df984e42773da)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:00.940052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3342e49c-217d-46ee-abb1-fd0d5863c14a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:00.969916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 321/627] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:12:00 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/c3cf87f0515b460dad8c0ff5591c2685/roles/d60298c676854e83b88df984e42773da => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:12:00.972769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342e49c-217d-46ee-abb1-fd0d5863c14a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:01.150107 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:01.151715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6722fa52-c6ca-4d06-add8-c5a1bbafc896 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:01.258045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3342e49c-217d-46ee-abb1-fd0d5863c14a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:01.268098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb67e-394f-4fdf-ab68-e663580cfd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:01.269664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb67e-394f-4fdf-ab68-e663580cfd67 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:01.270415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb67e-394f-4fdf-ab68-e663580cfd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:01.271236 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb67e-394f-4fdf-ab68-e663580cfd67 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:01.272553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afb67e-394f-4fdf-ab68-e663580cfd67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:01.273371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afb67e-394f-4fdf-ab68-e663580cfd67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:01.276445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afb67e-394f-4fdf-ab68-e663580cfd67 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:01.293537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 307/628] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:01.307205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb67e-394f-4fdf-ab68-e663580cfd67 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:01.336378 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f00d32-6d78-47b8-81f2-ccaabbbc26cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:01.337231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f00d32-6d78-47b8-81f2-ccaabbbc26cd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:01.337603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f00d32-6d78-47b8-81f2-ccaabbbc26cd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:01.338238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f00d32-6d78-47b8-81f2-ccaabbbc26cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:01.357631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f00d32-6d78-47b8-81f2-ccaabbbc26cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-b2f00d32-6d78-47b8-81f2-ccaabbbc26cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:01.370956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:01.372826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:01.374277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 322/629] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:12:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:01.383166 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f00d32-6d78-47b8-81f2-ccaabbbc26cd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:01.437914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:01.508556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb67e-394f-4fdf-ab68-e663580cfd67 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:01.587681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c276b-e9e9-40db-80c5-f67bf469342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:01.589088 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c276b-e9e9-40db-80c5-f67bf469342d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:01.589592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c276b-e9e9-40db-80c5-f67bf469342d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:01.590093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c276b-e9e9-40db-80c5-f67bf469342d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:01.623919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c276b-e9e9-40db-80c5-f67bf469342d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:01.627136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c276b-e9e9-40db-80c5-f67bf469342d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:01.670673 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f00d32-6d78-47b8-81f2-ccaabbbc26cd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-8b7c276b-e9e9-40db-80c5-f67bf469342d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:01.679933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:01.681802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:01.683731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:01.683731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:01.683731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:01.683731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:01.683731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:01.683731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:01.683731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:01.683731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:01.683731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:12:01.683731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:01.697188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 308/630] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Jul 20 08:12:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 398 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:01.705571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c276b-e9e9-40db-80c5-f67bf469342d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:01.731169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:01.791468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f250a-c1e9-4e9b-a42f-fa8fb1e84ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:01.792241 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f250a-c1e9-4e9b-a42f-fa8fb1e84ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:01.792544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f250a-c1e9-4e9b-a42f-fa8fb1e84ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:01.792831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f250a-c1e9-4e9b-a42f-fa8fb1e84ce7 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:01.793655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f250a-c1e9-4e9b-a42f-fa8fb1e84ce7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:01.795342 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f250a-c1e9-4e9b-a42f-fa8fb1e84ce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:01.796322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f250a-c1e9-4e9b-a42f-fa8fb1e84ce7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:01.819367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 323/631] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:12:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:01.825630 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f250a-c1e9-4e9b-a42f-fa8fb1e84ce7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:01.937714 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c276b-e9e9-40db-80c5-f67bf469342d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:01.998517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:02.006322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f250a-c1e9-4e9b-a42f-fa8fb1e84ce7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:02.015700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7eaca-5f92-45b8-a3b0-f4ae755f18d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:02.018095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7eaca-5f92-45b8-a3b0-f4ae755f18d7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:02.019141 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7eaca-5f92-45b8-a3b0-f4ae755f18d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:02.019876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7eaca-5f92-45b8-a3b0-f4ae755f18d7 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:02.021722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7eaca-5f92-45b8-a3b0-f4ae755f18d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:02.022506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7eaca-5f92-45b8-a3b0-f4ae755f18d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:02.026033 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7eaca-5f92-45b8-a3b0-f4ae755f18d7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:02.062790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c083c83-8abe-48c2-b1d4-d18e8b3a5f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:02.063848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c083c83-8abe-48c2-b1d4-d18e8b3a5f11 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:02.064189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 309/632] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Jul 20 08:12:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:02.064347 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c083c83-8abe-48c2-b1d4-d18e8b3a5f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:02.064674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c083c83-8abe-48c2-b1d4-d18e8b3a5f11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:02.066900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c4459-4f7b-49f0-b048-b0cbfb493479 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:02.067426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c4459-4f7b-49f0-b048-b0cbfb493479 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:02.067848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c4459-4f7b-49f0-b048-b0cbfb493479 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:02.070664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 310/633] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:02.074547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c4459-4f7b-49f0-b048-b0cbfb493479 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:02.081758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c083c83-8abe-48c2-b1d4-d18e8b3a5f11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:02.085004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c083c83-8abe-48c2-b1d4-d18e8b3a5f11 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4c083c83-8abe-48c2-b1d4-d18e8b3a5f11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:02.120190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:02.121303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:02.122649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:02.122649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:02.122649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:02.122649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:02.122649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 324/634] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:02.128096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92e0f-f006-4ad8-adc4-d94c2a80e928 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:02.128096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92e0f-f006-4ad8-adc4-d94c2a80e928 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:02.128096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92e0f-f006-4ad8-adc4-d94c2a80e928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:02.153059 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b2f92e0f-f006-4ad8-adc4-d94c2a80e928 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:02.154527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b2f92e0f-f006-4ad8-adc4-d94c2a80e928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:02.189691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:02.244089 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c4459-4f7b-49f0-b048-b0cbfb493479 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:02.291693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 325/635] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:02.293886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f92e0f-f006-4ad8-adc4-d94c2a80e928 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:02.326342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a71a6a-9881-4efc-8267-05f8a3a8069a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:02.327476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a71a6a-9881-4efc-8267-05f8a3a8069a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:02.328024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a71a6a-9881-4efc-8267-05f8a3a8069a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:02.328548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a71a6a-9881-4efc-8267-05f8a3a8069a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:02.444496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:02.471262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a71a6a-9881-4efc-8267-05f8a3a8069a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:02.473737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a71a6a-9881-4efc-8267-05f8a3a8069a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:02.483014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f92e0f-f006-4ad8-adc4-d94c2a80e928 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:02.543426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fe11b-5aa6-4884-830c-8833e38cf0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:02.544484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fe11b-5aa6-4884-830c-8833e38cf0d9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:02.544944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fe11b-5aa6-4884-830c-8833e38cf0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:02.545394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fe11b-5aa6-4884-830c-8833e38cf0d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:02.546443 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fe11b-5aa6-4884-830c-8833e38cf0d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:02.547053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fe11b-5aa6-4884-830c-8833e38cf0d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:02.549323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fe11b-5aa6-4884-830c-8833e38cf0d9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:02.570353 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 326/636] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:02.571966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fe11b-5aa6-4884-830c-8833e38cf0d9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:02.599933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:02.652313 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 311/637] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:12:02.658810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12242434-44cd-428e-af7c-4fa21ae05a28 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:02.659678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12242434-44cd-428e-af7c-4fa21ae05a28 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:02.660429 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12242434-44cd-428e-af7c-4fa21ae05a28 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:02.668838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 312/638] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:02 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:02.670127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12242434-44cd-428e-af7c-4fa21ae05a28 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:02.778296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fe11b-5aa6-4884-830c-8833e38cf0d9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:02.850968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:02.858104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6427c-6b97-4424-bf46-0697a31b60da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:02.863126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6427c-6b97-4424-bf46-0697a31b60da None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:02.863576 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6427c-6b97-4424-bf46-0697a31b60da None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:02.864023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6427c-6b97-4424-bf46-0697a31b60da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:02.881252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12242434-44cd-428e-af7c-4fa21ae05a28 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:02.892792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6427c-6b97-4424-bf46-0697a31b60da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-30b6427c-6b97-4424-bf46-0697a31b60da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:02.913421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:02.914780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:02.915614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:02.916248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 327/639] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:12:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:02.918557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6427c-6b97-4424-bf46-0697a31b60da None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:02.937706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a94ba-0cde-4d2a-b290-f899622bb6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:02.939673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a94ba-0cde-4d2a-b290-f899622bb6af None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:02.940235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a94ba-0cde-4d2a-b290-f899622bb6af None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:02.940837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a94ba-0cde-4d2a-b290-f899622bb6af None admin] PATH_INFO: `/v3/users/glance` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:02.951397 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a94ba-0cde-4d2a-b290-f899622bb6af None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:02.953248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a94ba-0cde-4d2a-b290-f899622bb6af None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e58a94ba-0cde-4d2a-b290-f899622bb6af None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:02.974529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:02.975607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:02.977189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:02.977189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:02.977189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:02.977189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:02.977189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:02.977189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:02.977189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:02.977189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:02.977189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 08:12:02.977189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:02.980092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 313/640] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:12:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:02.981998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41d1bb-f38e-4bf5-a99b-37727c235444 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:02.982493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41d1bb-f38e-4bf5-a99b-37727c235444 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:02.983072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41d1bb-f38e-4bf5-a99b-37727c235444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:03.005055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2e41d1bb-f38e-4bf5-a99b-37727c235444 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:03.006737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2e41d1bb-f38e-4bf5-a99b-37727c235444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:03.092544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:03.138068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6427c-6b97-4424-bf46-0697a31b60da None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:03.166651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 314/641] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:03.166651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41d1bb-f38e-4bf5-a99b-37727c235444 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:03.234914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76ae3f-0d31-4b8a-bd30-863e1f019e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:03.238056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76ae3f-0d31-4b8a-bd30-863e1f019e82 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:03.238770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76ae3f-0d31-4b8a-bd30-863e1f019e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:03.239492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76ae3f-0d31-4b8a-bd30-863e1f019e82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:03.270203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76ae3f-0d31-4b8a-bd30-863e1f019e82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:03.274222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76ae3f-0d31-4b8a-bd30-863e1f019e82 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ad76ae3f-0d31-4b8a-bd30-863e1f019e82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:03.310636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:03.312549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:03.314946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:03.314946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:03.314946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:03.314946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:03.327642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 328/642] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:03.327642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76ae3f-0d31-4b8a-bd30-863e1f019e82 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:03.339563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:03.381414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41d1bb-f38e-4bf5-a99b-37727c235444 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:03.448105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40efb3-62e6-471e-bf60-eb639f0a0933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:03.449173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40efb3-62e6-471e-bf60-eb639f0a0933 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:03.449695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40efb3-62e6-471e-bf60-eb639f0a0933 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:03.450201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40efb3-62e6-471e-bf60-eb639f0a0933 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:03.451726 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40efb3-62e6-471e-bf60-eb639f0a0933 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:03.452285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40efb3-62e6-471e-bf60-eb639f0a0933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:03.455006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40efb3-62e6-471e-bf60-eb639f0a0933 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:03.473912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 315/643] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:12:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:03.475165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40efb3-62e6-471e-bf60-eb639f0a0933 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:03.520299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76ae3f-0d31-4b8a-bd30-863e1f019e82 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:03.565535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:03.583534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2ca55-4565-4cae-946a-b5a54ea4c455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:03.584423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2ca55-4565-4cae-946a-b5a54ea4c455 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:03.584750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2ca55-4565-4cae-946a-b5a54ea4c455 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:03.585076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2ca55-4565-4cae-946a-b5a54ea4c455 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:03.586164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2ca55-4565-4cae-946a-b5a54ea4c455 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:03.586498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2ca55-4565-4cae-946a-b5a54ea4c455 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:03.588147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2ca55-4565-4cae-946a-b5a54ea4c455 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:03.616853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 329/644] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Jul 20 08:12:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:03.617737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2ca55-4565-4cae-946a-b5a54ea4c455 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:03.683979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40efb3-62e6-471e-bf60-eb639f0a0933 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:03.754055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db41e3d-b414-4c3c-8a7a-6b080ad8ab16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:03.755409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db41e3d-b414-4c3c-8a7a-6b080ad8ab16 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:03.755903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db41e3d-b414-4c3c-8a7a-6b080ad8ab16 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:03.756386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db41e3d-b414-4c3c-8a7a-6b080ad8ab16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:03.770993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2ca55-4565-4cae-946a-b5a54ea4c455 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:03.771804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:03.778934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db41e3d-b414-4c3c-8a7a-6b080ad8ab16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-1db41e3d-b414-4c3c-8a7a-6b080ad8ab16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:03.795744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:03.797237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:03.799066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:03.802917 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 316/645] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:12:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:03.804210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db41e3d-b414-4c3c-8a7a-6b080ad8ab16 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:03.830570 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52496eec-2c85-494e-81aa-1c9bbc966c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:03.831770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52496eec-2c85-494e-81aa-1c9bbc966c57 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:03.832290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52496eec-2c85-494e-81aa-1c9bbc966c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:03.832803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52496eec-2c85-494e-81aa-1c9bbc966c57 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:03.834221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52496eec-2c85-494e-81aa-1c9bbc966c57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:03.834823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52496eec-2c85-494e-81aa-1c9bbc966c57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:03.837500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52496eec-2c85-494e-81aa-1c9bbc966c57 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:03.854677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 330/646] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:03.855635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52496eec-2c85-494e-81aa-1c9bbc966c57 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:03.924316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:03.960112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db41e3d-b414-4c3c-8a7a-6b080ad8ab16 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:04.016571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d432be5-663b-4867-9bf4-cdc3d2983009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:04.017706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d432be5-663b-4867-9bf4-cdc3d2983009 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:04.018132 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d432be5-663b-4867-9bf4-cdc3d2983009 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:04.018462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d432be5-663b-4867-9bf4-cdc3d2983009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:04.023931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52496eec-2c85-494e-81aa-1c9bbc966c57 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:04.036727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d432be5-663b-4867-9bf4-cdc3d2983009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:04.039018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d432be5-663b-4867-9bf4-cdc3d2983009 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9d432be5-663b-4867-9bf4-cdc3d2983009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:04.065830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:04.067967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:04.070443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:04.070443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:04.070443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:04.070443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:04.070443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 317/647] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:04.070443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d432be5-663b-4867-9bf4-cdc3d2983009 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:04.091755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94922bf3-7795-4071-b8e8-6a2e7bca4b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:04.092515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94922bf3-7795-4071-b8e8-6a2e7bca4b51 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:04.092868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94922bf3-7795-4071-b8e8-6a2e7bca4b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:04.093278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94922bf3-7795-4071-b8e8-6a2e7bca4b51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:04.154223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:04.190666 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94922bf3-7795-4071-b8e8-6a2e7bca4b51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:04.192453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94922bf3-7795-4071-b8e8-6a2e7bca4b51 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:04.232265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d432be5-663b-4867-9bf4-cdc3d2983009 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:04.255511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:04.294842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72d809-1e48-400c-b9a4-34e824e28606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:04.295616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72d809-1e48-400c-b9a4-34e824e28606 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:04.295885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72d809-1e48-400c-b9a4-34e824e28606 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:04.296288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c72d809-1e48-400c-b9a4-34e824e28606 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:04.297335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72d809-1e48-400c-b9a4-34e824e28606 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:04.297875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72d809-1e48-400c-b9a4-34e824e28606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:04.300960 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c72d809-1e48-400c-b9a4-34e824e28606 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:04.316884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 318/648] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:12:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:04.317835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72d809-1e48-400c-b9a4-34e824e28606 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:04.346003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 331/649] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:12:04.348195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7afdc-682a-432a-9c30-37796f6468cb None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:04.348679 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7afdc-682a-432a-9c30-37796f6468cb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:04.349150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f7afdc-682a-432a-9c30-37796f6468cb None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:04.354643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 332/650] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:04.354643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7afdc-682a-432a-9c30-37796f6468cb None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:04.477514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c72d809-1e48-400c-b9a4-34e824e28606 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:04.493029 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f7afdc-682a-432a-9c30-37796f6468cb None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:04.506361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:04.547876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cb0f9-ff75-4bc6-9127-ee499f39fbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:04.549292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb0f9-ff75-4bc6-9127-ee499f39fbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:04.549626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb0f9-ff75-4bc6-9127-ee499f39fbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:04.549957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb0f9-ff75-4bc6-9127-ee499f39fbf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:04.553488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b070e0f-c342-4f15-8d06-a822650d3d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:04.554164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b070e0f-c342-4f15-8d06-a822650d3d3e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:04.554445 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b070e0f-c342-4f15-8d06-a822650d3d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:04.554757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b070e0f-c342-4f15-8d06-a822650d3d3e None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:04.555499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b070e0f-c342-4f15-8d06-a822650d3d3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:04.555804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b070e0f-c342-4f15-8d06-a822650d3d3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:04.557212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b070e0f-c342-4f15-8d06-a822650d3d3e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:04.559851 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cb0f9-ff75-4bc6-9127-ee499f39fbf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:04.562210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cb0f9-ff75-4bc6-9127-ee499f39fbf5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:04.571061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 333/651] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:04.571931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b070e0f-c342-4f15-8d06-a822650d3d3e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:04.576063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d87cb0f9-ff75-4bc6-9127-ee499f39fbf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:04.591722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:04.592874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:04.593782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:04.593782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:04.593782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:04.593782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:04.593782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:04.593782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:04.593782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:04.593782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:04.593782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 319/652] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:04.595581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cb0f9-ff75-4bc6-9127-ee499f39fbf5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:04.706398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b070e0f-c342-4f15-8d06-a822650d3d3e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:04.744169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cb0f9-ff75-4bc6-9127-ee499f39fbf5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:04.745718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cec38d-ff51-4e66-bdb8-2d8eefc3a328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:04.746525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cec38d-ff51-4e66-bdb8-2d8eefc3a328 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:04.746964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cec38d-ff51-4e66-bdb8-2d8eefc3a328 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:04.747243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cec38d-ff51-4e66-bdb8-2d8eefc3a328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:04.759602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cec38d-ff51-4e66-bdb8-2d8eefc3a328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:04.761115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cec38d-ff51-4e66-bdb8-2d8eefc3a328 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-62cec38d-ff51-4e66-bdb8-2d8eefc3a328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:04.781253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:04.783324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:04.785181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:04.785181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:04.785181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:04.785181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:04.785181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 334/653] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:04.785181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d491d-e8c2-48ed-a4d0-f4edc2c2fb92 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:04.785603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d491d-e8c2-48ed-a4d0-f4edc2c2fb92 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:04.785603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d491d-e8c2-48ed-a4d0-f4edc2c2fb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:04.792640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304657b-55ae-4db3-8c2b-40430dc1e09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:04.794053 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304657b-55ae-4db3-8c2b-40430dc1e09a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:04.794563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304657b-55ae-4db3-8c2b-40430dc1e09a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:04.795118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304657b-55ae-4db3-8c2b-40430dc1e09a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:04.804064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-959d491d-e8c2-48ed-a4d0-f4edc2c2fb92 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:04.805211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-959d491d-e8c2-48ed-a4d0-f4edc2c2fb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:04.815720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304657b-55ae-4db3-8c2b-40430dc1e09a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:04.818553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304657b-55ae-4db3-8c2b-40430dc1e09a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b304657b-55ae-4db3-8c2b-40430dc1e09a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:04.852943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:04.854491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:04.856386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:04.856386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:04.856386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:04.856386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:04.856386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 320/654] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:04.857642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304657b-55ae-4db3-8c2b-40430dc1e09a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:04.871248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:04.910767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 335/655] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:04.910767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d491d-e8c2-48ed-a4d0-f4edc2c2fb92 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:04.915157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:05.025329 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304657b-55ae-4db3-8c2b-40430dc1e09a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:05.047781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d491d-e8c2-48ed-a4d0-f4edc2c2fb92 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:05.072494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2452992-6460-4101-bcf7-c9d9dc6cee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:05.073684 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2452992-6460-4101-bcf7-c9d9dc6cee9e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:05.074119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2452992-6460-4101-bcf7-c9d9dc6cee9e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:05.074605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2452992-6460-4101-bcf7-c9d9dc6cee9e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:05.085563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2452992-6460-4101-bcf7-c9d9dc6cee9e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:05.087181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2452992-6460-4101-bcf7-c9d9dc6cee9e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:05.095735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a40f82-0e5b-4d12-b58b-fbc4239dcd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:05.096692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a40f82-0e5b-4d12-b58b-fbc4239dcd4d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:05.097105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a40f82-0e5b-4d12-b58b-fbc4239dcd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:05.097385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a40f82-0e5b-4d12-b58b-fbc4239dcd4d None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:05.099081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a40f82-0e5b-4d12-b58b-fbc4239dcd4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:05.099397 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a40f82-0e5b-4d12-b58b-fbc4239dcd4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:05.100905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a40f82-0e5b-4d12-b58b-fbc4239dcd4d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e2452992-6460-4101-bcf7-c9d9dc6cee9e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:05.102815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:05.103936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:05.105005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:05.105005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:05.105005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:05.105005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:05.105005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:05.105005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:05.105005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:05.105005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:05.105005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 08:12:05.105005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:05.105005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 321/656] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:12:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:05.106193 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2452992-6460-4101-bcf7-c9d9dc6cee9e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:05.123388 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 336/657] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:05.125125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a40f82-0e5b-4d12-b58b-fbc4239dcd4d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:05.201802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:05.241648 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:05.267503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2452992-6460-4101-bcf7-c9d9dc6cee9e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:05.291526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a40f82-0e5b-4d12-b58b-fbc4239dcd4d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:05.311649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88678b-1f8d-449c-91d0-e56da0b269b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:05.312340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88678b-1f8d-449c-91d0-e56da0b269b2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:05.312627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88678b-1f8d-449c-91d0-e56da0b269b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:05.312914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88678b-1f8d-449c-91d0-e56da0b269b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:05.314375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88678b-1f8d-449c-91d0-e56da0b269b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:05.314757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88678b-1f8d-449c-91d0-e56da0b269b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:05.316177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88678b-1f8d-449c-91d0-e56da0b269b2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:05.333930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 322/658] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:05.334814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88678b-1f8d-449c-91d0-e56da0b269b2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:05.340282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d325bc8-14e1-4200-8c10-ad89f84e7b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:05.341343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d325bc8-14e1-4200-8c10-ad89f84e7b39 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:05.341806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d325bc8-14e1-4200-8c10-ad89f84e7b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:05.342259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d325bc8-14e1-4200-8c10-ad89f84e7b39 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:05.343577 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d325bc8-14e1-4200-8c10-ad89f84e7b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:05.344088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d325bc8-14e1-4200-8c10-ad89f84e7b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:05.347234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d325bc8-14e1-4200-8c10-ad89f84e7b39 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:05.369540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 337/659] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:05.370774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d325bc8-14e1-4200-8c10-ad89f84e7b39 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:05.488646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88678b-1f8d-449c-91d0-e56da0b269b2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:05.512728 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d325bc8-14e1-4200-8c10-ad89f84e7b39 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:05.523821 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:05.551420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d94bcd-9d05-4082-84c5-f41da88a856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:05.552522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d94bcd-9d05-4082-84c5-f41da88a856c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:05.552907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d94bcd-9d05-4082-84c5-f41da88a856c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:05.553280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d94bcd-9d05-4082-84c5-f41da88a856c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:05.560656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16fa35-2276-4379-ab06-ae4d7f581229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:05.560969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16fa35-2276-4379-ab06-ae4d7f581229 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:05.562302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16fa35-2276-4379-ab06-ae4d7f581229 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:05.562302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16fa35-2276-4379-ab06-ae4d7f581229 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:05.562302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16fa35-2276-4379-ab06-ae4d7f581229 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:05.562707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16fa35-2276-4379-ab06-ae4d7f581229 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:05.564752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16fa35-2276-4379-ab06-ae4d7f581229 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:05.570358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d94bcd-9d05-4082-84c5-f41da88a856c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-a1d94bcd-9d05-4082-84c5-f41da88a856c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:05.578803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:05.580839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 323/660] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:12:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:05.582230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d94bcd-9d05-4082-84c5-f41da88a856c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:05.590563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 338/661] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Jul 20 08:12:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:05.591222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16fa35-2276-4379-ab06-ae4d7f581229 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:05.599735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:05.730147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16fa35-2276-4379-ab06-ae4d7f581229 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:05.754552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d94bcd-9d05-4082-84c5-f41da88a856c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:05.780431 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff454e3-9de5-4b92-8100-6dd24c77ff2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:05.781848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff454e3-9de5-4b92-8100-6dd24c77ff2e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:05.783031 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff454e3-9de5-4b92-8100-6dd24c77ff2e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:05.784060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff454e3-9de5-4b92-8100-6dd24c77ff2e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:05.797560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff454e3-9de5-4b92-8100-6dd24c77ff2e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:05.799667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff454e3-9de5-4b92-8100-6dd24c77ff2e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-1ff454e3-9de5-4b92-8100-6dd24c77ff2e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:05.816543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:05.818492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:05.821848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:05.821848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:05.821848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:05.821848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:05.821848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:05.821848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:05.821848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:05.821848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:05.821848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 08:12:05.821848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:05.821848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 339/662] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:12:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:05.823650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990be9ed-cf2e-4aff-bcb2-21c61272dfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:05.823838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff454e3-9de5-4b92-8100-6dd24c77ff2e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:05.823996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990be9ed-cf2e-4aff-bcb2-21c61272dfcf None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:05.823996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990be9ed-cf2e-4aff-bcb2-21c61272dfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:05.823996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990be9ed-cf2e-4aff-bcb2-21c61272dfcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:05.823996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990be9ed-cf2e-4aff-bcb2-21c61272dfcf None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=84f0da8709624834976d0662aeb02ea6) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:05.823996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990be9ed-cf2e-4aff-bcb2-21c61272dfcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:05.827357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990be9ed-cf2e-4aff-bcb2-21c61272dfcf None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:05.838869 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 324/663] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:05 2022] GET /identity/v3/role_assignments?user.id=84f0da8709624834976d0662aeb02ea6&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:05.840175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990be9ed-cf2e-4aff-bcb2-21c61272dfcf None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:05.858509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:05.986883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff454e3-9de5-4b92-8100-6dd24c77ff2e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:05.988136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:06.051981 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990be9ed-cf2e-4aff-bcb2-21c61272dfcf None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:06.055724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7209033-31be-4d63-a8a4-7d231797a170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:06.056619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7209033-31be-4d63-a8a4-7d231797a170 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:06.061628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7209033-31be-4d63-a8a4-7d231797a170 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:06.061628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7209033-31be-4d63-a8a4-7d231797a170 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:06.072011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7209033-31be-4d63-a8a4-7d231797a170 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:06.074566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7209033-31be-4d63-a8a4-7d231797a170 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d7209033-31be-4d63-a8a4-7d231797a170 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:06.103362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:06.105054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:06.106774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:06.106774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:06.106774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:06.106774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:06.106774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:06.106774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:06.106774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:06.106774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:06.106774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 340/664] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:06.106774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7209033-31be-4d63-a8a4-7d231797a170 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:06.124875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a77862-cfb1-4d7e-8bbc-f79aafcdfa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:06.125484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a77862-cfb1-4d7e-8bbc-f79aafcdfa54 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:06.125770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a77862-cfb1-4d7e-8bbc-f79aafcdfa54 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:06.126078 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a77862-cfb1-4d7e-8bbc-f79aafcdfa54 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:06.127355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a77862-cfb1-4d7e-8bbc-f79aafcdfa54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:06.127355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a77862-cfb1-4d7e-8bbc-f79aafcdfa54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:06.128775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a77862-cfb1-4d7e-8bbc-f79aafcdfa54 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:06.142142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 325/665] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:12:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:06.144150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a77862-cfb1-4d7e-8bbc-f79aafcdfa54 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:06.243392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:06.271649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7209033-31be-4d63-a8a4-7d231797a170 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:06.283952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a77862-cfb1-4d7e-8bbc-f79aafcdfa54 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:06.331300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:06.331565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bc479-6b3f-4731-a260-c62995ac55c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:06.332730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bc479-6b3f-4731-a260-c62995ac55c0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:06.332939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bc479-6b3f-4731-a260-c62995ac55c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:06.333477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bc479-6b3f-4731-a260-c62995ac55c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:06.337837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bee1d-1fde-4854-90e6-abe7be138cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:06.338532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bee1d-1fde-4854-90e6-abe7be138cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:06.338858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bee1d-1fde-4854-90e6-abe7be138cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:06.339143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bee1d-1fde-4854-90e6-abe7be138cf4 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:06.340685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bee1d-1fde-4854-90e6-abe7be138cf4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:06.340685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bee1d-1fde-4854-90e6-abe7be138cf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:06.341879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bee1d-1fde-4854-90e6-abe7be138cf4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:06.370657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 326/666] 10.222.0.64 () {56 vars in 1139 bytes} [Wed Jul 20 08:12:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:06.370657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bee1d-1fde-4854-90e6-abe7be138cf4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:06.455365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bc479-6b3f-4731-a260-c62995ac55c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:06.456914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7bc479-6b3f-4731-a260-c62995ac55c0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:06.515586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bee1d-1fde-4854-90e6-abe7be138cf4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:06.576617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ae261-f160-411e-822e-f6910168624e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:06.577683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ae261-f160-411e-822e-f6910168624e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:06.577972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ae261-f160-411e-822e-f6910168624e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:06.578251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ae261-f160-411e-822e-f6910168624e None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:06.579563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ae261-f160-411e-822e-f6910168624e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:06.580433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ae261-f160-411e-822e-f6910168624e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:06.581733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ae261-f160-411e-822e-f6910168624e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:06.582726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:06.603899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 327/667] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:06.605225 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ae261-f160-411e-822e-f6910168624e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:06.623962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 341/668] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:12:06.623962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bc479-6b3f-4731-a260-c62995ac55c0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:06.686062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:06.770272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ae261-f160-411e-822e-f6910168624e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:06.784421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7bc479-6b3f-4731-a260-c62995ac55c0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:06.821373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fc6f8-4ec4-43fe-b2f3-6ae2de1c1164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:06.822568 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fc6f8-4ec4-43fe-b2f3-6ae2de1c1164 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:06.823042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fc6f8-4ec4-43fe-b2f3-6ae2de1c1164 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:06.823519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621fc6f8-4ec4-43fe-b2f3-6ae2de1c1164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:06.833466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e457d-e16e-414f-8a4e-d31aac77c787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:06.834243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e457d-e16e-414f-8a4e-d31aac77c787 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:06.834539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e457d-e16e-414f-8a4e-d31aac77c787 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:06.834836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e457d-e16e-414f-8a4e-d31aac77c787 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:06.839395 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fc6f8-4ec4-43fe-b2f3-6ae2de1c1164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:06.841583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621fc6f8-4ec4-43fe-b2f3-6ae2de1c1164 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-621fc6f8-4ec4-43fe-b2f3-6ae2de1c1164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:06.865795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:06.866952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:06.867894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:06.867894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:06.867894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:06.867894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:06.873376 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 328/669] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:06.874515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fc6f8-4ec4-43fe-b2f3-6ae2de1c1164 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:06.910184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:06.933293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e457d-e16e-414f-8a4e-d31aac77c787 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:06.934822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e457d-e16e-414f-8a4e-d31aac77c787 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:07.032089 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621fc6f8-4ec4-43fe-b2f3-6ae2de1c1164 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:07.034418 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:07.084239 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 342/670] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:12:07.084239 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e457d-e16e-414f-8a4e-d31aac77c787 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:07.088752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1a8e7-8e05-40b2-8d38-ff67ff743593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:07.089808 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1a8e7-8e05-40b2-8d38-ff67ff743593 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:07.090275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1a8e7-8e05-40b2-8d38-ff67ff743593 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:07.090772 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b1a8e7-8e05-40b2-8d38-ff67ff743593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:07.092243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1a8e7-8e05-40b2-8d38-ff67ff743593 None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=3778997232d44cd3a3c4fb3d53e4db53) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:07.092764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1a8e7-8e05-40b2-8d38-ff67ff743593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:07.096041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b1a8e7-8e05-40b2-8d38-ff67ff743593 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:07.106136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 329/671] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:06 2022] GET /identity/v3/role_assignments?user.id=3778997232d44cd3a3c4fb3d53e4db53&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:07.107688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1a8e7-8e05-40b2-8d38-ff67ff743593 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:07.218851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:07.244898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e457d-e16e-414f-8a4e-d31aac77c787 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:07.293376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49078919-4031-4774-aa65-9d4b669fec9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:07.294325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49078919-4031-4774-aa65-9d4b669fec9e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:07.294747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49078919-4031-4774-aa65-9d4b669fec9e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:07.295135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49078919-4031-4774-aa65-9d4b669fec9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:07.316092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49078919-4031-4774-aa65-9d4b669fec9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:07.319568 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b1a8e7-8e05-40b2-8d38-ff67ff743593 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:07.320434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49078919-4031-4774-aa65-9d4b669fec9e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-49078919-4031-4774-aa65-9d4b669fec9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:07.346157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:07.347844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:07.349547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:07.349547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:07.349547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:07.349547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:07.352762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 343/672] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:07.353959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49078919-4031-4774-aa65-9d4b669fec9e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:07.392551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6887f13-ad76-41f7-9460-b8dff40758e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:07.394885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6887f13-ad76-41f7-9460-b8dff40758e3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:07.395621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6887f13-ad76-41f7-9460-b8dff40758e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:07.396543 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6887f13-ad76-41f7-9460-b8dff40758e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:07.398215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6887f13-ad76-41f7-9460-b8dff40758e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:07.399041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6887f13-ad76-41f7-9460-b8dff40758e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:07.402995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6887f13-ad76-41f7-9460-b8dff40758e3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:07.426617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 330/673] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:07.427412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6887f13-ad76-41f7-9460-b8dff40758e3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:07.473193 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:07.550649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49078919-4031-4774-aa65-9d4b669fec9e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:07.594466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6887f13-ad76-41f7-9460-b8dff40758e3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:07.603302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dad9c0-c75a-4334-8cb2-540e22a2b501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:07.604119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dad9c0-c75a-4334-8cb2-540e22a2b501 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:07.604410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dad9c0-c75a-4334-8cb2-540e22a2b501 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:07.605411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dad9c0-c75a-4334-8cb2-540e22a2b501 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:07.609305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:07.621826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dad9c0-c75a-4334-8cb2-540e22a2b501 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:07.624044 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dad9c0-c75a-4334-8cb2-540e22a2b501 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:07.644595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ad133-da40-4c9b-90b2-d6c965ffb2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:07.645853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad133-da40-4c9b-90b2-d6c965ffb2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:07.646455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad133-da40-4c9b-90b2-d6c965ffb2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:07.647088 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ad133-da40-4c9b-90b2-d6c965ffb2a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:07.648575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ad133-da40-4c9b-90b2-d6c965ffb2a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:07.649301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ad133-da40-4c9b-90b2-d6c965ffb2a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-56dad9c0-c75a-4334-8cb2-540e22a2b501 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:07.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:07.651302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:07.653094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:07.653094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:07.653094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:07.653094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:07.653540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ad133-da40-4c9b-90b2-d6c965ffb2a3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:07.655961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 344/674] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:07.658609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dad9c0-c75a-4334-8cb2-540e22a2b501 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:07.676297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 331/675] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:07.678047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ad133-da40-4c9b-90b2-d6c965ffb2a3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:07.796776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:07.806569 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dad9c0-c75a-4334-8cb2-540e22a2b501 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:07.819815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ad133-da40-4c9b-90b2-d6c965ffb2a3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:07.859557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc34533-9588-4a11-ae87-8da3227e2bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:07.860839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc34533-9588-4a11-ae87-8da3227e2bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:07.861267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc34533-9588-4a11-ae87-8da3227e2bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:07.861604 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc34533-9588-4a11-ae87-8da3227e2bf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:07.862452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc34533-9588-4a11-ae87-8da3227e2bf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:07.862902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc34533-9588-4a11-ae87-8da3227e2bf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:07.864381 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd58fc-2144-4840-8848-dc11ce64ab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:07.865091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc34533-9588-4a11-ae87-8da3227e2bf9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:07.866251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd58fc-2144-4840-8848-dc11ce64ab0f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:07.867161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd58fc-2144-4840-8848-dc11ce64ab0f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:07.867879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd58fc-2144-4840-8848-dc11ce64ab0f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:07.882798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 332/676] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:07.885290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc34533-9588-4a11-ae87-8da3227e2bf9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:07.885960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd58fc-2144-4840-8848-dc11ce64ab0f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:07.894942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd58fc-2144-4840-8848-dc11ce64ab0f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-67cd58fc-2144-4840-8848-dc11ce64ab0f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:07.924820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:07.925966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:07.927909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:07.927909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:07.927909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:07.927909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:07.927909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:07.927909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:07.927909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:07.927909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:07.927909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:12:07.927909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:07.939555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 345/677] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:12:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:07.941701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd58fc-2144-4840-8848-dc11ce64ab0f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:08.023271 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc34533-9588-4a11-ae87-8da3227e2bf9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:08.043824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:08.088814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303aa144-5717-49eb-a15a-0fe4722b7ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:08.090457 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303aa144-5717-49eb-a15a-0fe4722b7ebf None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:08.091125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303aa144-5717-49eb-a15a-0fe4722b7ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:08.092617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303aa144-5717-49eb-a15a-0fe4722b7ebf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:08.112295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303aa144-5717-49eb-a15a-0fe4722b7ebf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:08.114520 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303aa144-5717-49eb-a15a-0fe4722b7ebf None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-303aa144-5717-49eb-a15a-0fe4722b7ebf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:08.138299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:08.139546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:08.140401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:08.140401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:08.140401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:08.140401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:08.140401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:08.140401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:08.140401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:08.140401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:08.146206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 333/678] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:08.147156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd58fc-2144-4840-8848-dc11ce64ab0f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:08.147897 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303aa144-5717-49eb-a15a-0fe4722b7ebf None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:08.168392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:08.206014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e46375-99a0-46b7-956d-d58ae97b4b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:08.207510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e46375-99a0-46b7-956d-d58ae97b4b28 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:08.208270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e46375-99a0-46b7-956d-d58ae97b4b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:08.208653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e46375-99a0-46b7-956d-d58ae97b4b28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:08.220020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e46375-99a0-46b7-956d-d58ae97b4b28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:08.222180 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e46375-99a0-46b7-956d-d58ae97b4b28 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e9e46375-99a0-46b7-956d-d58ae97b4b28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:08.249216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:08.250817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:08.252937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:08.252937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:08.252937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:08.252937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:08.252937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:08.252937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:08.252937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:08.252937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:08.252937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 346/679] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:08.252937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e46375-99a0-46b7-956d-d58ae97b4b28 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:08.332414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303aa144-5717-49eb-a15a-0fe4722b7ebf None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:08.381558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e954e4-9a09-4547-8240-ec7f4656a0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:08.382192 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e954e4-9a09-4547-8240-ec7f4656a0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:08.382477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e954e4-9a09-4547-8240-ec7f4656a0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:08.382783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e954e4-9a09-4547-8240-ec7f4656a0f4 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:08.384239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e954e4-9a09-4547-8240-ec7f4656a0f4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:08.384611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e954e4-9a09-4547-8240-ec7f4656a0f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:08.387154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e954e4-9a09-4547-8240-ec7f4656a0f4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:08.421955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 334/680] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Jul 20 08:12:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:08.421955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e954e4-9a09-4547-8240-ec7f4656a0f4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:08.457858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e46375-99a0-46b7-956d-d58ae97b4b28 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:08.470788 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:08.508201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f61da-830d-4bcd-a4d7-6ac0dc90f60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:08.509017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f61da-830d-4bcd-a4d7-6ac0dc90f60f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:08.509302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f61da-830d-4bcd-a4d7-6ac0dc90f60f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:08.509581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f61da-830d-4bcd-a4d7-6ac0dc90f60f None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:08.510408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f61da-830d-4bcd-a4d7-6ac0dc90f60f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:08.510750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f61da-830d-4bcd-a4d7-6ac0dc90f60f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:08.512234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f61da-830d-4bcd-a4d7-6ac0dc90f60f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:08.517853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:08.526212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 347/681] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:08.527643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecc7d3-74d7-4533-bf59-0686a0321a33 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:08.527928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecc7d3-74d7-4533-bf59-0686a0321a33 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:08.528203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecc7d3-74d7-4533-bf59-0686a0321a33 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:08.534480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 348/682] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:08.534480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecc7d3-74d7-4533-bf59-0686a0321a33 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:08.578976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e954e4-9a09-4547-8240-ec7f4656a0f4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:08.623692 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057b3e2-5623-4559-ad69-a9fd65d88524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:08.624495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057b3e2-5623-4559-ad69-a9fd65d88524 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:08.624783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057b3e2-5623-4559-ad69-a9fd65d88524 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:08.625067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057b3e2-5623-4559-ad69-a9fd65d88524 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:08.625876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057b3e2-5623-4559-ad69-a9fd65d88524 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:08.626900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057b3e2-5623-4559-ad69-a9fd65d88524 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:08.628200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057b3e2-5623-4559-ad69-a9fd65d88524 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:08.646691 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 335/683] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:08.646691 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057b3e2-5623-4559-ad69-a9fd65d88524 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:08.700510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecc7d3-74d7-4533-bf59-0686a0321a33 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:08.752017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cf72b-35c7-4df0-8623-15ad0abc08a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:08.753587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cf72b-35c7-4df0-8623-15ad0abc08a2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:08.754091 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cf72b-35c7-4df0-8623-15ad0abc08a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:08.754461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cf72b-35c7-4df0-8623-15ad0abc08a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:08.785538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057b3e2-5623-4559-ad69-a9fd65d88524 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:08.807874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:08.822200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:08.838169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34878edc-9d9f-4780-866d-5284bd83c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:08.839263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878edc-9d9f-4780-866d-5284bd83c992 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:08.839609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878edc-9d9f-4780-866d-5284bd83c992 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:08.839980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34878edc-9d9f-4780-866d-5284bd83c992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:08.840961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34878edc-9d9f-4780-866d-5284bd83c992 None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=41a647bd72f344fa963dcd3eb5434b6e) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:08.841342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34878edc-9d9f-4780-866d-5284bd83c992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:08.843042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34878edc-9d9f-4780-866d-5284bd83c992 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:08.851618 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 336/684] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:08 2022] GET /identity/v3/role_assignments?user.id=41a647bd72f344fa963dcd3eb5434b6e&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:08.853956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099be8c-6cdf-4002-88e6-9d30188e4a69 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:08.854513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099be8c-6cdf-4002-88e6-9d30188e4a69 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:08.855034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099be8c-6cdf-4002-88e6-9d30188e4a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:08.867953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cf72b-35c7-4df0-8623-15ad0abc08a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:08.870020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2cf72b-35c7-4df0-8623-15ad0abc08a2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:08.888139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3099be8c-6cdf-4002-88e6-9d30188e4a69 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:08.890872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3099be8c-6cdf-4002-88e6-9d30188e4a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:09.042069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 337/685] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:09.043632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099be8c-6cdf-4002-88e6-9d30188e4a69 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:09.043929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 349/686] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:12:09.053653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cf72b-35c7-4df0-8623-15ad0abc08a2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:09.181977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:09.223159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099be8c-6cdf-4002-88e6-9d30188e4a69 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:09.243678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cf72b-35c7-4df0-8623-15ad0abc08a2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:09.245278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:09.272627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c232db-c57e-4984-ade3-7e28d85cc426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:09.273814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c232db-c57e-4984-ade3-7e28d85cc426 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:09.274269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c232db-c57e-4984-ade3-7e28d85cc426 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:09.275313 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c232db-c57e-4984-ade3-7e28d85cc426 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:09.276452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c232db-c57e-4984-ade3-7e28d85cc426 None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=a3b03383554346138b98becb2828851a) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:09.277288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c232db-c57e-4984-ade3-7e28d85cc426 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:09.279094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c232db-c57e-4984-ade3-7e28d85cc426 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:09.288497 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 338/687] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:09 2022] GET /identity/v3/role_assignments?user.id=a3b03383554346138b98becb2828851a&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:09.290037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c232db-c57e-4984-ade3-7e28d85cc426 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:09.316002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174c8e0-72dd-4d6e-99bc-6cd86e8a8626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:09.317646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174c8e0-72dd-4d6e-99bc-6cd86e8a8626 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:09.318330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174c8e0-72dd-4d6e-99bc-6cd86e8a8626 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:09.318545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174c8e0-72dd-4d6e-99bc-6cd86e8a8626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:09.338376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174c8e0-72dd-4d6e-99bc-6cd86e8a8626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:09.340846 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174c8e0-72dd-4d6e-99bc-6cd86e8a8626 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5174c8e0-72dd-4d6e-99bc-6cd86e8a8626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:09.388744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:09.389914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:09.390838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:09.390838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:09.390838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:09.390838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:09.398139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 350/688] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:09.399744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174c8e0-72dd-4d6e-99bc-6cd86e8a8626 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:09.438864 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c232db-c57e-4984-ade3-7e28d85cc426 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:09.497757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffea12-e17f-49e5-a775-7ba348cfc27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:09.499004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffea12-e17f-49e5-a775-7ba348cfc27f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:09.500104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffea12-e17f-49e5-a775-7ba348cfc27f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:09.500739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ffea12-e17f-49e5-a775-7ba348cfc27f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:09.515328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ffea12-e17f-49e5-a775-7ba348cfc27f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-33ffea12-e17f-49e5-a775-7ba348cfc27f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:09.526363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:09.527590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:09.528433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:09.534162 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 339/689] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:12:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:09.544735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffea12-e17f-49e5-a775-7ba348cfc27f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:09.630848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:09.677810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174c8e0-72dd-4d6e-99bc-6cd86e8a8626 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:09.703026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:09.724845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad954a-7e72-4fa3-bf63-843a6572ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:09.725728 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad954a-7e72-4fa3-bf63-843a6572ed86 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:09.726020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad954a-7e72-4fa3-bf63-843a6572ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:09.726322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad954a-7e72-4fa3-bf63-843a6572ed86 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:09.727590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad954a-7e72-4fa3-bf63-843a6572ed86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:09.727971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad954a-7e72-4fa3-bf63-843a6572ed86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:09.729604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad954a-7e72-4fa3-bf63-843a6572ed86 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:09.745314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 351/690] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:09.751037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad954a-7e72-4fa3-bf63-843a6572ed86 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:09.776941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ffea12-e17f-49e5-a775-7ba348cfc27f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:09.879206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad954a-7e72-4fa3-bf63-843a6572ed86 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:09.886660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a545989-7f7e-4eaa-9e1a-5dec79efd7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:09.888174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a545989-7f7e-4eaa-9e1a-5dec79efd7bf None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:09.888762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a545989-7f7e-4eaa-9e1a-5dec79efd7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:09.889273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a545989-7f7e-4eaa-9e1a-5dec79efd7bf None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:09.890274 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a545989-7f7e-4eaa-9e1a-5dec79efd7bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:09.890824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a545989-7f7e-4eaa-9e1a-5dec79efd7bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:09.893292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a545989-7f7e-4eaa-9e1a-5dec79efd7bf None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:09.903754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 340/691] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:12:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:09.910034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a545989-7f7e-4eaa-9e1a-5dec79efd7bf None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:09.920094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126774d-5570-407b-92af-6c54fbe902d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:09.920927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126774d-5570-407b-92af-6c54fbe902d7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:09.921218 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126774d-5570-407b-92af-6c54fbe902d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:09.921507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126774d-5570-407b-92af-6c54fbe902d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:09.930033 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126774d-5570-407b-92af-6c54fbe902d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:09.931629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126774d-5570-407b-92af-6c54fbe902d7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-9126774d-5570-407b-92af-6c54fbe902d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:09.948237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:09.949280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:09.950136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:09.950136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:09.950136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:09.950136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:09.950136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:09.950136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:09.950136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:09.950136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:09.950136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 352/692] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:09.956352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126774d-5570-407b-92af-6c54fbe902d7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:10.039997 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:10.076577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a545989-7f7e-4eaa-9e1a-5dec79efd7bf None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:10.081646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:10.141107 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126774d-5570-407b-92af-6c54fbe902d7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:10.147306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690aeac-0919-45a9-96a1-b6fec6e58d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:10.148685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690aeac-0919-45a9-96a1-b6fec6e58d57 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:10.149195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690aeac-0919-45a9-96a1-b6fec6e58d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:10.149650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690aeac-0919-45a9-96a1-b6fec6e58d57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:10.169062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690aeac-0919-45a9-96a1-b6fec6e58d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:10.171898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690aeac-0919-45a9-96a1-b6fec6e58d57 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:10.187846 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45886502-6053-49dc-9add-b8e5c039835d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:10.190433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45886502-6053-49dc-9add-b8e5c039835d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:10.190989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45886502-6053-49dc-9add-b8e5c039835d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:10.191795 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45886502-6053-49dc-9add-b8e5c039835d None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:10.192831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45886502-6053-49dc-9add-b8e5c039835d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:10.193349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45886502-6053-49dc-9add-b8e5c039835d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:10.195270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45886502-6053-49dc-9add-b8e5c039835d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-3690aeac-0919-45a9-96a1-b6fec6e58d57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:10.208229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:10.209169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:10.210045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:10.210045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:10.210045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:10.210045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:10.212711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 353/693] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:10.214392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf96ed-15f8-436b-b87a-358bb62eed68 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:10.214922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf96ed-15f8-436b-b87a-358bb62eed68 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:10.215408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf96ed-15f8-436b-b87a-358bb62eed68 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:10.217293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 354/694] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:10.218765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 341/695] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:10.221670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf96ed-15f8-436b-b87a-358bb62eed68 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:10.223827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146dcfa-8d7a-48c4-b173-a291efa27185 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:10.224751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146dcfa-8d7a-48c4-b173-a291efa27185 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:10.225239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146dcfa-8d7a-48c4-b173-a291efa27185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:10.248932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2146dcfa-8d7a-48c4-b173-a291efa27185 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:10.251301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2146dcfa-8d7a-48c4-b173-a291efa27185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:10.365429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:10.370882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 342/696] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:10.372094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146dcfa-8d7a-48c4-b173-a291efa27185 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:10.372603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf96ed-15f8-436b-b87a-358bb62eed68 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:10.421318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa8da3-0805-4e29-b8d2-410dc567d56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:10.423794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa8da3-0805-4e29-b8d2-410dc567d56b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:10.424324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa8da3-0805-4e29-b8d2-410dc567d56b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:10.424808 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa8da3-0805-4e29-b8d2-410dc567d56b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:10.426321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa8da3-0805-4e29-b8d2-410dc567d56b None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=c3cf87f0515b460dad8c0ff5591c2685) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:10.426908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa8da3-0805-4e29-b8d2-410dc567d56b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:10.429912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa8da3-0805-4e29-b8d2-410dc567d56b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:10.439239 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 355/697] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:10 2022] GET /identity/v3/role_assignments?user.id=c3cf87f0515b460dad8c0ff5591c2685&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:10.440479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa8da3-0805-4e29-b8d2-410dc567d56b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:10.627279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:10.678743 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2146dcfa-8d7a-48c4-b173-a291efa27185 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:10.756347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb9f35-8dfd-4a37-95d3-84cc9ad21fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:10.757446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb9f35-8dfd-4a37-95d3-84cc9ad21fef None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:10.758016 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb9f35-8dfd-4a37-95d3-84cc9ad21fef None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:10.758504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb9f35-8dfd-4a37-95d3-84cc9ad21fef None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:10.760093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb9f35-8dfd-4a37-95d3-84cc9ad21fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:10.761884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb9f35-8dfd-4a37-95d3-84cc9ad21fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:10.762739 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa8da3-0805-4e29-b8d2-410dc567d56b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:10.764177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb9f35-8dfd-4a37-95d3-84cc9ad21fef None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:10.793973 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 343/698] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:10.802526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb9f35-8dfd-4a37-95d3-84cc9ad21fef None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:10.848552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717eeb7-ef7d-4808-af2a-1b843a4d8d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:10.850621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717eeb7-ef7d-4808-af2a-1b843a4d8d6e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:10.850955 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717eeb7-ef7d-4808-af2a-1b843a4d8d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:10.851276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717eeb7-ef7d-4808-af2a-1b843a4d8d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:10.878225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717eeb7-ef7d-4808-af2a-1b843a4d8d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:10.881653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717eeb7-ef7d-4808-af2a-1b843a4d8d6e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-f717eeb7-ef7d-4808-af2a-1b843a4d8d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:10.909408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:10.910409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:10.911541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:10.911541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:10.911541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:10.911541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:10.912004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 356/699] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:10.918435 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717eeb7-ef7d-4808-af2a-1b843a4d8d6e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:10.948628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:10.976229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb9f35-8dfd-4a37-95d3-84cc9ad21fef None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:11.031780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98321ec6-c7f5-4a54-b283-23b71dd05adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:11.033041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98321ec6-c7f5-4a54-b283-23b71dd05adb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:11.033330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98321ec6-c7f5-4a54-b283-23b71dd05adb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:11.033606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98321ec6-c7f5-4a54-b283-23b71dd05adb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:11.044896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98321ec6-c7f5-4a54-b283-23b71dd05adb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:11.046388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98321ec6-c7f5-4a54-b283-23b71dd05adb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-98321ec6-c7f5-4a54-b283-23b71dd05adb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:11.063508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:11.064536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:11.065458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:11.065458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:11.065458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:11.065458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:11.065458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:11.065458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:11.065458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:11.065458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:11.065458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:12:11.065458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:11.066171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 344/700] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Jul 20 08:12:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:11.075889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98321ec6-c7f5-4a54-b283-23b71dd05adb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:11.090260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:11.101942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717eeb7-ef7d-4808-af2a-1b843a4d8d6e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:11.176122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782c51b-cf96-4e1d-bbd5-f2ccd8ec8be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:11.177232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782c51b-cf96-4e1d-bbd5-f2ccd8ec8be5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:11.177718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782c51b-cf96-4e1d-bbd5-f2ccd8ec8be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:11.178301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782c51b-cf96-4e1d-bbd5-f2ccd8ec8be5 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:11.179440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782c51b-cf96-4e1d-bbd5-f2ccd8ec8be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:11.180047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782c51b-cf96-4e1d-bbd5-f2ccd8ec8be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:11.181783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782c51b-cf96-4e1d-bbd5-f2ccd8ec8be5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:11.189368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98321ec6-c7f5-4a54-b283-23b71dd05adb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:11.202864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 357/701] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:11.205448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782c51b-cf96-4e1d-bbd5-f2ccd8ec8be5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:11.281851 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1506f44-6008-4fbc-8857-7795ba2b81b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:11.284091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1506f44-6008-4fbc-8857-7795ba2b81b0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:11.285617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1506f44-6008-4fbc-8857-7795ba2b81b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:11.287030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1506f44-6008-4fbc-8857-7795ba2b81b0 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:11.288184 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1506f44-6008-4fbc-8857-7795ba2b81b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:11.290893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1506f44-6008-4fbc-8857-7795ba2b81b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:11.293178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1506f44-6008-4fbc-8857-7795ba2b81b0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:11.327755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 345/702] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Jul 20 08:12:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:11.334696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1506f44-6008-4fbc-8857-7795ba2b81b0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:11.414385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:11.473785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782c51b-cf96-4e1d-bbd5-f2ccd8ec8be5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:11.477479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:11.542876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536a0ab-8e86-4fb4-be1f-4cdc5e54a14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:11.542876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536a0ab-8e86-4fb4-be1f-4cdc5e54a14d None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:11.542876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536a0ab-8e86-4fb4-be1f-4cdc5e54a14d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:11.542876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536a0ab-8e86-4fb4-be1f-4cdc5e54a14d None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:11.543309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536a0ab-8e86-4fb4-be1f-4cdc5e54a14d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:11.543309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536a0ab-8e86-4fb4-be1f-4cdc5e54a14d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:11.584452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 358/703] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:12:11 2022] POST /identity/v3/users => generated 254 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:11.588855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1506f44-6008-4fbc-8857-7795ba2b81b0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:11.633897 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b43cc-748e-4689-ac12-364bc27191d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:11.634660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b43cc-748e-4689-ac12-364bc27191d1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:11.635032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b43cc-748e-4689-ac12-364bc27191d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:11.635428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b43cc-748e-4689-ac12-364bc27191d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:11.733406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b43cc-748e-4689-ac12-364bc27191d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:11.734837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b43cc-748e-4689-ac12-364bc27191d1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:11.830966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:11.890023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d068692-82f7-41b1-b24c-9e4888b38f08 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:11.890376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d068692-82f7-41b1-b24c-9e4888b38f08 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:11.890606 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d068692-82f7-41b1-b24c-9e4888b38f08 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:11.892807 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 359/704] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:12:11.895126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 346/705] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:12:11.900677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d068692-82f7-41b1-b24c-9e4888b38f08 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:11.906019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfce540-fc5e-4bc6-a360-659b85e273c6 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:11.906380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfce540-fc5e-4bc6-a360-659b85e273c6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:11.906644 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfce540-fc5e-4bc6-a360-659b85e273c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:11.930624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3dfce540-fc5e-4bc6-a360-659b85e273c6 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:11.932082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3dfce540-fc5e-4bc6-a360-659b85e273c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:12.049041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 347/706] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 20 08:12:12.080682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfce540-fc5e-4bc6-a360-659b85e273c6 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:12.104486 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d068692-82f7-41b1-b24c-9e4888b38f08 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:12.149892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:12.159659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43632993-11e3-4702-b4f3-581d69a6a885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:12.160625 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632993-11e3-4702-b4f3-581d69a6a885 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:12.160794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632993-11e3-4702-b4f3-581d69a6a885 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:12.161844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43632993-11e3-4702-b4f3-581d69a6a885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:12.175763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43632993-11e3-4702-b4f3-581d69a6a885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:12.178715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43632993-11e3-4702-b4f3-581d69a6a885 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-43632993-11e3-4702-b4f3-581d69a6a885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:12.205600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:12.207803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:12.209698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:12.209698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:12.209698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:12.209698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:12.209698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 360/707] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:12.211382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b839b13-99c7-443b-befa-67624053904e None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:12.211794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b839b13-99c7-443b-befa-67624053904e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:12.212172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b839b13-99c7-443b-befa-67624053904e None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:12.213946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 361/708] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:12.218993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b839b13-99c7-443b-befa-67624053904e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:12.258366 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:12.271160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfce540-fc5e-4bc6-a360-659b85e273c6 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:12.330312 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e7a124-bd10-44ff-b1e6-4e7d331ecbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:12.331549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7a124-bd10-44ff-b1e6-4e7d331ecbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:12.332063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7a124-bd10-44ff-b1e6-4e7d331ecbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:12.332805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e7a124-bd10-44ff-b1e6-4e7d331ecbb4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:12.333705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7a124-bd10-44ff-b1e6-4e7d331ecbb4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:12.337023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e7a124-bd10-44ff-b1e6-4e7d331ecbb4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-f5e7a124-bd10-44ff-b1e6-4e7d331ecbb4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:12.356551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:12.357720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 08:12:12.358675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:12.363233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 348/709] 10.222.0.64 () {56 vars in 1129 bytes} [Wed Jul 20 08:12:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:12.365118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a1a09-c0e2-483d-b576-66c6e4f62f9d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:12.365603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a1a09-c0e2-483d-b576-66c6e4f62f9d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:12.366719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a1a09-c0e2-483d-b576-66c6e4f62f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:12.395374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f35a1a09-c0e2-483d-b576-66c6e4f62f9d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:12.395374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f35a1a09-c0e2-483d-b576-66c6e4f62f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:12.409160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b839b13-99c7-443b-befa-67624053904e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:12.466333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c163694-e0d7-44bb-9baa-429d282b505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:12.468076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c163694-e0d7-44bb-9baa-429d282b505e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:12.468601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c163694-e0d7-44bb-9baa-429d282b505e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:12.469080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c163694-e0d7-44bb-9baa-429d282b505e None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:12.471924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c163694-e0d7-44bb-9baa-429d282b505e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:12.472500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c163694-e0d7-44bb-9baa-429d282b505e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:12.474688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c163694-e0d7-44bb-9baa-429d282b505e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:12.493919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 362/710] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:12.496196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c163694-e0d7-44bb-9baa-429d282b505e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:12.535277 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 349/711] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:12.537261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a1a09-c0e2-483d-b576-66c6e4f62f9d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:12.604881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:12.701469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c163694-e0d7-44bb-9baa-429d282b505e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:12.761698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd12b-d77d-4027-a567-e430e7aa8856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:12.762550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd12b-d77d-4027-a567-e430e7aa8856 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:12.762739 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd12b-d77d-4027-a567-e430e7aa8856 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:12.763025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd12b-d77d-4027-a567-e430e7aa8856 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:12.763799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fd12b-d77d-4027-a567-e430e7aa8856 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:12.764102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fd12b-d77d-4027-a567-e430e7aa8856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:12.765575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fd12b-d77d-4027-a567-e430e7aa8856 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:12.772705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 363/712] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Jul 20 08:12:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:12.773653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd12b-d77d-4027-a567-e430e7aa8856 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:12.784683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:12.809201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a1a09-c0e2-483d-b576-66c6e4f62f9d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:12.857493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75ca43-56ad-40b6-a70d-fd48f7563625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:12.858729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75ca43-56ad-40b6-a70d-fd48f7563625 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:12.859475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75ca43-56ad-40b6-a70d-fd48f7563625 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:12.859988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75ca43-56ad-40b6-a70d-fd48f7563625 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:12.870846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75ca43-56ad-40b6-a70d-fd48f7563625 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:12.872697 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75ca43-56ad-40b6-a70d-fd48f7563625 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ba75ca43-56ad-40b6-a70d-fd48f7563625 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:12.892482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:12.893948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:12.894841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:12.894841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:12.894841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:12.894841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:12.894841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:12.894841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:12.894841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:12.894841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:12.902560 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 350/713] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:12.903620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75ca43-56ad-40b6-a70d-fd48f7563625 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:12.973665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd12b-d77d-4027-a567-e430e7aa8856 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:12.985784 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:13.047190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b58d0-5f1b-4268-ab55-891ba193bce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:13.049857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b58d0-5f1b-4268-ab55-891ba193bce0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:13.050723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b58d0-5f1b-4268-ab55-891ba193bce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:13.051052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b58d0-5f1b-4268-ab55-891ba193bce0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:13.051769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b58d0-5f1b-4268-ab55-891ba193bce0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:13.053308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b58d0-5f1b-4268-ab55-891ba193bce0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:13.070428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75ca43-56ad-40b6-a70d-fd48f7563625 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-458b58d0-5f1b-4268-ab55-891ba193bce0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:13.086611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:13.088343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 08:12:13.089750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:13.092070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 364/714] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:12 2022] GET /identity/v3/services/network => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:13.099899 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b58d0-5f1b-4268-ab55-891ba193bce0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:13.130619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc129bf-2ea4-4e1d-b2fb-602b39cbeb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:13.131149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc129bf-2ea4-4e1d-b2fb-602b39cbeb12 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:13.131434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc129bf-2ea4-4e1d-b2fb-602b39cbeb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:13.131711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc129bf-2ea4-4e1d-b2fb-602b39cbeb12 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:13.132515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc129bf-2ea4-4e1d-b2fb-602b39cbeb12 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:13.132822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc129bf-2ea4-4e1d-b2fb-602b39cbeb12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:13.134284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc129bf-2ea4-4e1d-b2fb-602b39cbeb12 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:13.142315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 351/715] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Jul 20 08:12:12 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:13.143425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc129bf-2ea4-4e1d-b2fb-602b39cbeb12 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:13.165578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:13.322528 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b58d0-5f1b-4268-ab55-891ba193bce0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:13.329560 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc129bf-2ea4-4e1d-b2fb-602b39cbeb12 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:13.379450 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce517c6-6a13-4332-8ed7-bb4d710c6ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:13.380151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce517c6-6a13-4332-8ed7-bb4d710c6ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:13.380519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce517c6-6a13-4332-8ed7-bb4d710c6ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:13.380772 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce517c6-6a13-4332-8ed7-bb4d710c6ac8 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:13.381565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce517c6-6a13-4332-8ed7-bb4d710c6ac8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:13.381910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce517c6-6a13-4332-8ed7-bb4d710c6ac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:13.383614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce517c6-6a13-4332-8ed7-bb4d710c6ac8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:13.385754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9693e63-6187-4361-a054-841e69c42a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:13.386801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9693e63-6187-4361-a054-841e69c42a6f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:13.387433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9693e63-6187-4361-a054-841e69c42a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:13.387914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9693e63-6187-4361-a054-841e69c42a6f None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:13.388926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9693e63-6187-4361-a054-841e69c42a6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:13.389445 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9693e63-6187-4361-a054-841e69c42a6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:13.394677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 352/716] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:13.395825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9693e63-6187-4361-a054-841e69c42a6f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:13.402264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce517c6-6a13-4332-8ed7-bb4d710c6ac8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:13.413223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 365/717] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:13.419447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9693e63-6187-4361-a054-841e69c42a6f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:13.468935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:13.527020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:13.560768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce517c6-6a13-4332-8ed7-bb4d710c6ac8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:13.572752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9693e63-6187-4361-a054-841e69c42a6f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:13.619157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b76ca-312e-4bf1-a3d9-905bbf6ba9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:13.620162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf976cd1-f0c9-4d2c-b001-7823eb87d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:13.620581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b76ca-312e-4bf1-a3d9-905bbf6ba9dd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:13.620971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976cd1-f0c9-4d2c-b001-7823eb87d1be None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:13.621303 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b76ca-312e-4bf1-a3d9-905bbf6ba9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:13.621430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976cd1-f0c9-4d2c-b001-7823eb87d1be None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:13.621648 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf976cd1-f0c9-4d2c-b001-7823eb87d1be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:13.622078 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b76ca-312e-4bf1-a3d9-905bbf6ba9dd None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:13.622643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf976cd1-f0c9-4d2c-b001-7823eb87d1be None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=84f0da8709624834976d0662aeb02ea6) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:13.623025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf976cd1-f0c9-4d2c-b001-7823eb87d1be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:13.623384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b76ca-312e-4bf1-a3d9-905bbf6ba9dd None admin] RBAC: Adding query filter params (type=network) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:13.623888 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b76ca-312e-4bf1-a3d9-905bbf6ba9dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:13.624749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf976cd1-f0c9-4d2c-b001-7823eb87d1be None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:13.625645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b76ca-312e-4bf1-a3d9-905bbf6ba9dd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:13.632439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 366/718] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:13 2022] GET /identity/v3/role_assignments?user.id=84f0da8709624834976d0662aeb02ea6&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:13.633314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 353/719] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:13.687961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af286a1-3303-4f2e-a636-8442ec3a24f1 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:13.689018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af286a1-3303-4f2e-a636-8442ec3a24f1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:13.689487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af286a1-3303-4f2e-a636-8442ec3a24f1 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:13.692572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 367/720] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:13.702110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde91f-9d2f-4730-81c8-aa404208a429 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:13.702615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde91f-9d2f-4730-81c8-aa404208a429 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:13.703152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fde91f-9d2f-4730-81c8-aa404208a429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:13.727816 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-92fde91f-9d2f-4730-81c8-aa404208a429 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:13.729489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-92fde91f-9d2f-4730-81c8-aa404208a429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:13.846788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 354/721] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:13.920271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af286a1-3303-4f2e-a636-8442ec3a24f1 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:14.086533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:14.100093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af286a1-3303-4f2e-a636-8442ec3a24f1 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:14.167449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcec106-b31f-410e-bb5b-93fa0217dcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:14.168402 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcec106-b31f-410e-bb5b-93fa0217dcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:14.168732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcec106-b31f-410e-bb5b-93fa0217dcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:14.169004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcec106-b31f-410e-bb5b-93fa0217dcc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:14.184788 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcec106-b31f-410e-bb5b-93fa0217dcc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-2fcec106-b31f-410e-bb5b-93fa0217dcc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:14.193132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:14.195608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:14.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 368/722] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:12:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:14.208063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde91f-9d2f-4730-81c8-aa404208a429 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:14.354896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:14.446730 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fde91f-9d2f-4730-81c8-aa404208a429 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:14.523665 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374cdd6-364b-4b56-9412-da5435645b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:14.524965 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374cdd6-364b-4b56-9412-da5435645b89 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:14.525476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374cdd6-364b-4b56-9412-da5435645b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:14.525939 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374cdd6-364b-4b56-9412-da5435645b89 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:14.527410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374cdd6-364b-4b56-9412-da5435645b89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:14.528044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374cdd6-364b-4b56-9412-da5435645b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:14.530728 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374cdd6-364b-4b56-9412-da5435645b89 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:14.549701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 355/723] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:12:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:14.559029 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcec106-b31f-410e-bb5b-93fa0217dcc6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:14.687080 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa14137-bf16-409c-8099-588d4865180b None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:14.687730 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa14137-bf16-409c-8099-588d4865180b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:14.688337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa14137-bf16-409c-8099-588d4865180b None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:14.692755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 356/724] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:14.700112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6624512-aafe-4af4-a5cb-4909fda7dc68 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:14.701400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6624512-aafe-4af4-a5cb-4909fda7dc68 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:14.701400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6624512-aafe-4af4-a5cb-4909fda7dc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:14.743074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f6624512-aafe-4af4-a5cb-4909fda7dc68 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:14.744453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f6624512-aafe-4af4-a5cb-4909fda7dc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:14.883198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcec106-b31f-410e-bb5b-93fa0217dcc6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:14.954877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 357/725] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:14.983273 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ef54e-5949-4dae-a65f-8b5a07eb6c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:14.984688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ef54e-5949-4dae-a65f-8b5a07eb6c2b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:14.986027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ef54e-5949-4dae-a65f-8b5a07eb6c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:14.986542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ef54e-5949-4dae-a65f-8b5a07eb6c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:15.009294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ef54e-5949-4dae-a65f-8b5a07eb6c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:15.012589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6624512-aafe-4af4-a5cb-4909fda7dc68 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:15.013793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ef54e-5949-4dae-a65f-8b5a07eb6c2b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5a1ef54e-5949-4dae-a65f-8b5a07eb6c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:15.057490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:15.058476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:15.059437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:15.059437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:15.059437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:15.059437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:15.061799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 369/726] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 504 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:15.071332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ef54e-5949-4dae-a65f-8b5a07eb6c2b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:15.093658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:15.112938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:15.268315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6624512-aafe-4af4-a5cb-4909fda7dc68 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:15.324730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ef54e-5949-4dae-a65f-8b5a07eb6c2b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:15.405207 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fae499-f6d2-4336-a992-7675dc94da65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:15.406685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fae499-f6d2-4336-a992-7675dc94da65 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:15.408024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fae499-f6d2-4336-a992-7675dc94da65 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:15.408738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fae499-f6d2-4336-a992-7675dc94da65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:15.432146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fae499-f6d2-4336-a992-7675dc94da65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:15.435712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e225caf-082b-4d22-b16e-c92cb026298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:15.438709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e225caf-082b-4d22-b16e-c92cb026298e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:15.439724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e225caf-082b-4d22-b16e-c92cb026298e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:15.440480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e225caf-082b-4d22-b16e-c92cb026298e None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:15.442090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e225caf-082b-4d22-b16e-c92cb026298e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:15.442871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e225caf-082b-4d22-b16e-c92cb026298e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:15.445632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e225caf-082b-4d22-b16e-c92cb026298e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d0fae499-f6d2-4336-a992-7675dc94da65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:15.445843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:15.447753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:15.449710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:15.459627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 358/727] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:12:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:15.467261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fae499-f6d2-4336-a992-7675dc94da65 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:15.478919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 370/728] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:15.487557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e225caf-082b-4d22-b16e-c92cb026298e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:15.621988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:15.687355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e225caf-082b-4d22-b16e-c92cb026298e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:15.731591 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fae499-f6d2-4336-a992-7675dc94da65 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:15.738283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a10f7-6a96-43b2-84fd-239b3111987a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:15.739476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a10f7-6a96-43b2-84fd-239b3111987a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:15.739978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a10f7-6a96-43b2-84fd-239b3111987a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:15.740542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a10f7-6a96-43b2-84fd-239b3111987a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:15.757229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a10f7-6a96-43b2-84fd-239b3111987a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:15.759466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a10f7-6a96-43b2-84fd-239b3111987a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-3f0a10f7-6a96-43b2-84fd-239b3111987a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:15.788680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:15.789694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:15.790653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:15.790653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:15.790653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:15.790653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:15.790653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:15.790653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:15.790653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:15.790653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:15.790653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:12:15.790653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:15.793991 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:15.796373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 371/729] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:12:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:15.798373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a10f7-6a96-43b2-84fd-239b3111987a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:15.817545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16bd85-d1f3-4e2d-96e0-ffe0e0eabd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:15.818695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16bd85-d1f3-4e2d-96e0-ffe0e0eabd55 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:15.819191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16bd85-d1f3-4e2d-96e0-ffe0e0eabd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:15.819671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c16bd85-d1f3-4e2d-96e0-ffe0e0eabd55 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:15.820680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16bd85-d1f3-4e2d-96e0-ffe0e0eabd55 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:15.821222 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16bd85-d1f3-4e2d-96e0-ffe0e0eabd55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:15.822905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c16bd85-d1f3-4e2d-96e0-ffe0e0eabd55 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:15.844256 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 359/730] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:12:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:15.852214 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16bd85-d1f3-4e2d-96e0-ffe0e0eabd55 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:15.978370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a10f7-6a96-43b2-84fd-239b3111987a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:16.023036 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:16.028712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bfe51-5744-4045-bee6-a2c10d41a7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:16.030148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bfe51-5744-4045-bee6-a2c10d41a7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:16.030781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bfe51-5744-4045-bee6-a2c10d41a7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:16.031552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bfe51-5744-4045-bee6-a2c10d41a7a4 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:16.033014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bfe51-5744-4045-bee6-a2c10d41a7a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:16.034871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bfe51-5744-4045-bee6-a2c10d41a7a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:16.038765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bfe51-5744-4045-bee6-a2c10d41a7a4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:16.079961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c16bd85-d1f3-4e2d-96e0-ffe0e0eabd55 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:16.082671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 372/731] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Jul 20 08:12:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:16.093943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bfe51-5744-4045-bee6-a2c10d41a7a4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:16.152746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b93ca0-dcd5-45f6-bbde-6af0169c7c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:16.153630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b93ca0-dcd5-45f6-bbde-6af0169c7c33 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:16.153921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b93ca0-dcd5-45f6-bbde-6af0169c7c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:16.154210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b93ca0-dcd5-45f6-bbde-6af0169c7c33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:16.168211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b93ca0-dcd5-45f6-bbde-6af0169c7c33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:16.169702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b93ca0-dcd5-45f6-bbde-6af0169c7c33 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-73b93ca0-dcd5-45f6-bbde-6af0169c7c33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:16.200031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:16.200976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:16.201923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:16.201923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:16.201923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:16.201923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:16.201923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 360/732] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:16.211934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b93ca0-dcd5-45f6-bbde-6af0169c7c33 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:16.263716 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:16.321118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bfe51-5744-4045-bee6-a2c10d41a7a4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:16.371600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e69f9-3cec-458c-95a6-1d1c179cc0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:16.372729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e69f9-3cec-458c-95a6-1d1c179cc0f9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:16.373288 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e69f9-3cec-458c-95a6-1d1c179cc0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:16.373783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e69f9-3cec-458c-95a6-1d1c179cc0f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:16.382546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b93ca0-dcd5-45f6-bbde-6af0169c7c33 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:16.449675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47787079-72e9-447b-9092-e8b4a1b16e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:16.450785 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47787079-72e9-447b-9092-e8b4a1b16e51 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:16.451315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47787079-72e9-447b-9092-e8b4a1b16e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:16.451831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47787079-72e9-447b-9092-e8b4a1b16e51 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:16.452999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47787079-72e9-447b-9092-e8b4a1b16e51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:16.453579 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47787079-72e9-447b-9092-e8b4a1b16e51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:16.455438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47787079-72e9-447b-9092-e8b4a1b16e51 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:16.459575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:16.470831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 361/733] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:16.472657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b4236-ed3b-4050-9ecd-006f2a9861f5 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:16.473171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b4236-ed3b-4050-9ecd-006f2a9861f5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:16.473660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b4236-ed3b-4050-9ecd-006f2a9861f5 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:16.475307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 362/734] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:16.480639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b4236-ed3b-4050-9ecd-006f2a9861f5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:16.492820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e69f9-3cec-458c-95a6-1d1c179cc0f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:16.493680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e69f9-3cec-458c-95a6-1d1c179cc0f9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:16.617025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:16.627384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b4236-ed3b-4050-9ecd-006f2a9861f5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:16.681656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87369d58-ab3b-4949-b34d-64aa31a9ae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:16.682952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87369d58-ab3b-4949-b34d-64aa31a9ae20 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:16.683314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87369d58-ab3b-4949-b34d-64aa31a9ae20 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:16.683598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87369d58-ab3b-4949-b34d-64aa31a9ae20 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:16.694398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 373/735] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:12:16.695865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277c7a2-70fa-4acf-8b23-db6a1213b298 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:16.696338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277c7a2-70fa-4acf-8b23-db6a1213b298 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:16.696726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277c7a2-70fa-4acf-8b23-db6a1213b298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:16.698769 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87369d58-ab3b-4949-b34d-64aa31a9ae20 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:16.701013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87369d58-ab3b-4949-b34d-64aa31a9ae20 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:16.717486 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6277c7a2-70fa-4acf-8b23-db6a1213b298 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:16.719076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6277c7a2-70fa-4acf-8b23-db6a1213b298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-87369d58-ab3b-4949-b34d-64aa31a9ae20 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:16.721659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:16.723017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:16.724195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:16.724195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:16.724195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:16.724195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:16.724195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:16.724195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:16.724195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:16.724195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:16.724195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:16.724195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:16.724195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 363/736] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Jul 20 08:12:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:16.724844 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02229352-f1c0-4cdd-b533-6fe424323daa None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:16.724915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02229352-f1c0-4cdd-b533-6fe424323daa None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:16.725167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02229352-f1c0-4cdd-b533-6fe424323daa None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:16.726998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 364/737] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:16.728623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02229352-f1c0-4cdd-b533-6fe424323daa None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:16.846759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 374/738] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:16.847957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277c7a2-70fa-4acf-8b23-db6a1213b298 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:16.871138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:16.900368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02229352-f1c0-4cdd-b533-6fe424323daa None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:16.940655 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668fade-1816-4798-96fd-a0b32d213803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:16.941512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668fade-1816-4798-96fd-a0b32d213803 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:16.941779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668fade-1816-4798-96fd-a0b32d213803 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:16.942064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668fade-1816-4798-96fd-a0b32d213803 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:16.953565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668fade-1816-4798-96fd-a0b32d213803 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:16.955151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668fade-1816-4798-96fd-a0b32d213803 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-f668fade-1816-4798-96fd-a0b32d213803 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:16.975029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:16.976032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:16.976901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:16.976901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:16.976901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:16.976901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:16.976901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 365/739] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:16.982468 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5577b3-ac20-4d04-8036-b326ad286377 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:16.982468 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5577b3-ac20-4d04-8036-b326ad286377 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:16.982468 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5577b3-ac20-4d04-8036-b326ad286377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:16.982815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277c7a2-70fa-4acf-8b23-db6a1213b298 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:17.003070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-db5577b3-ac20-4d04-8036-b326ad286377 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:17.004370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-db5577b3-ac20-4d04-8036-b326ad286377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:17.006821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:17.030887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b79a0-5d51-45c2-b5f9-d9e390afd2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:17.032018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b79a0-5d51-45c2-b5f9-d9e390afd2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:17.032612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b79a0-5d51-45c2-b5f9-d9e390afd2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:17.033203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201b79a0-5d51-45c2-b5f9-d9e390afd2b5 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:17.034115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b79a0-5d51-45c2-b5f9-d9e390afd2b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:17.034612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b79a0-5d51-45c2-b5f9-d9e390afd2b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:17.036468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201b79a0-5d51-45c2-b5f9-d9e390afd2b5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:17.059391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 375/740] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Jul 20 08:12:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:17.060525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b79a0-5d51-45c2-b5f9-d9e390afd2b5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:17.090972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 366/741] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:17.092136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e41baa-612e-4a3f-ba48-21b29167e5d6 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:17.092426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e41baa-612e-4a3f-ba48-21b29167e5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:17.092703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e41baa-612e-4a3f-ba48-21b29167e5d6 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:17.098677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 367/742] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:17.098677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e41baa-612e-4a3f-ba48-21b29167e5d6 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:17.185523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:17.213206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e41baa-612e-4a3f-ba48-21b29167e5d6 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:17.226826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201b79a0-5d51-45c2-b5f9-d9e390afd2b5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:17.252624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99946df6-3cf7-446c-a1d3-abfca50587af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:17.253287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99946df6-3cf7-446c-a1d3-abfca50587af None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:17.253570 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99946df6-3cf7-446c-a1d3-abfca50587af None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:17.253849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99946df6-3cf7-446c-a1d3-abfca50587af None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:17.254571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99946df6-3cf7-446c-a1d3-abfca50587af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:17.254907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99946df6-3cf7-446c-a1d3-abfca50587af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:17.256302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99946df6-3cf7-446c-a1d3-abfca50587af None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:17.263152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:17.269757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 368/743] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:17.270605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99946df6-3cf7-446c-a1d3-abfca50587af None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:17.285596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77aca4-3c0d-4718-8b90-2d5676c6f50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:17.286292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77aca4-3c0d-4718-8b90-2d5676c6f50e None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:17.286602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77aca4-3c0d-4718-8b90-2d5676c6f50e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:17.286986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77aca4-3c0d-4718-8b90-2d5676c6f50e None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:17.287740 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77aca4-3c0d-4718-8b90-2d5676c6f50e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:17.289183 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77aca4-3c0d-4718-8b90-2d5676c6f50e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:17.305467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 376/744] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:12:17 2022] POST /identity/v3/services => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:17.306820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5639-bed9-4d59-805b-34cbd4e05c7b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:17.307112 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5639-bed9-4d59-805b-34cbd4e05c7b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:17.307386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5639-bed9-4d59-805b-34cbd4e05c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:17.327468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-44ac5639-bed9-4d59-805b-34cbd4e05c7b None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:17.328736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-44ac5639-bed9-4d59-805b-34cbd4e05c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:17.431363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 377/745] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 08:12:17.432856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac5639-bed9-4d59-805b-34cbd4e05c7b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:17.466108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99946df6-3cf7-446c-a1d3-abfca50587af None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:17.529291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cabf48-6a97-454e-99b7-7cc1726a77e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:17.529963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cabf48-6a97-454e-99b7-7cc1726a77e2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:17.530958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cabf48-6a97-454e-99b7-7cc1726a77e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:17.530958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cabf48-6a97-454e-99b7-7cc1726a77e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:17.533476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:17.611686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac5639-bed9-4d59-805b-34cbd4e05c7b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:17.636138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:17.644170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cabf48-6a97-454e-99b7-7cc1726a77e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:17.647599 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cabf48-6a97-454e-99b7-7cc1726a77e2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:17.660456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dce92-f9d6-45a2-a171-cb5e577e06b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:17.661305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dce92-f9d6-45a2-a171-cb5e577e06b6 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:17.661305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dce92-f9d6-45a2-a171-cb5e577e06b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:17.661305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dce92-f9d6-45a2-a171-cb5e577e06b6 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:17.662408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417dce92-f9d6-45a2-a171-cb5e577e06b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:17.664318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417dce92-f9d6-45a2-a171-cb5e577e06b6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:17.685125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 378/746] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:12:17 2022] POST /identity/v3/services => generated 240 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:17.686679 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dce92-f9d6-45a2-a171-cb5e577e06b6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:17.842770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 369/747] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:17 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 08:12:17.844050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cabf48-6a97-454e-99b7-7cc1726a77e2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:17.905322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dce92-f9d6-45a2-a171-cb5e577e06b6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:17.939697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:17.970562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b5cda-3b42-4bec-86c9-4a7d1abc8d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:17.971516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b5cda-3b42-4bec-86c9-4a7d1abc8d56 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:17.971888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b5cda-3b42-4bec-86c9-4a7d1abc8d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:17.972175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b5cda-3b42-4bec-86c9-4a7d1abc8d56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:17.979843 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b5cda-3b42-4bec-86c9-4a7d1abc8d56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:17.981341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b5cda-3b42-4bec-86c9-4a7d1abc8d56 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:17.994713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:18.000203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cabf48-6a97-454e-99b7-7cc1726a77e2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e10b5cda-3b42-4bec-86c9-4a7d1abc8d56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:18.001502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:18.002477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:18.003938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:18.003938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:18.003938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:18.003938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:18.003938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:18.003938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:18.003938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:18.003938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:18.003938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 379/748] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:18.004490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b5cda-3b42-4bec-86c9-4a7d1abc8d56 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:18.074369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d560417-2b26-41c2-8799-e7c7fd2bc3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:18.075205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d560417-2b26-41c2-8799-e7c7fd2bc3ce None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:18.075492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d560417-2b26-41c2-8799-e7c7fd2bc3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:18.075771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d560417-2b26-41c2-8799-e7c7fd2bc3ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:18.098774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d560417-2b26-41c2-8799-e7c7fd2bc3ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9d560417-2b26-41c2-8799-e7c7fd2bc3ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:18.114039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:18.115465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:18.116338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:18.120078 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 370/749] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:12:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:18.121651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d560417-2b26-41c2-8799-e7c7fd2bc3ce None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:18.145416 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b5cda-3b42-4bec-86c9-4a7d1abc8d56 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:18.200292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15794b-530e-4af1-847c-cc6cc8aedcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:18.201911 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15794b-530e-4af1-847c-cc6cc8aedcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:18.202546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15794b-530e-4af1-847c-cc6cc8aedcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:18.203206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15794b-530e-4af1-847c-cc6cc8aedcc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:18.224885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15794b-530e-4af1-847c-cc6cc8aedcc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:18.228110 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15794b-530e-4af1-847c-cc6cc8aedcc6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-7d15794b-530e-4af1-847c-cc6cc8aedcc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:18.264405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:18.265599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:18.267308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:18.267308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:18.267308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:18.267308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:18.271188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d560417-2b26-41c2-8799-e7c7fd2bc3ce None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:18.274898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 380/750] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:18.276614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15794b-530e-4af1-847c-cc6cc8aedcc6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:18.319447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:18.331605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af306860-4349-4a2f-a608-9c9b22ad01f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:18.332953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306860-4349-4a2f-a608-9c9b22ad01f2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:18.333600 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306860-4349-4a2f-a608-9c9b22ad01f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:18.334243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af306860-4349-4a2f-a608-9c9b22ad01f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:18.336054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af306860-4349-4a2f-a608-9c9b22ad01f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:18.336773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af306860-4349-4a2f-a608-9c9b22ad01f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:18.339935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af306860-4349-4a2f-a608-9c9b22ad01f2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:18.369885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 371/751] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:18.371758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af306860-4349-4a2f-a608-9c9b22ad01f2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:18.415355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:18.420652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15794b-530e-4af1-847c-cc6cc8aedcc6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:18.473553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf2795-ece6-4c7c-9e93-15f3b64f9230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:18.474810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf2795-ece6-4c7c-9e93-15f3b64f9230 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:18.475252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf2795-ece6-4c7c-9e93-15f3b64f9230 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:18.475704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf2795-ece6-4c7c-9e93-15f3b64f9230 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:18.477109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf2795-ece6-4c7c-9e93-15f3b64f9230 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:18.477581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf2795-ece6-4c7c-9e93-15f3b64f9230 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:18.479191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf2795-ece6-4c7c-9e93-15f3b64f9230 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:18.492823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 381/752] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:12:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:18.494015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf2795-ece6-4c7c-9e93-15f3b64f9230 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:18.589399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af306860-4349-4a2f-a608-9c9b22ad01f2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:18.650219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec569e-8510-4b0e-8edc-8ff74143830b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:18.651460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec569e-8510-4b0e-8edc-8ff74143830b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:18.652080 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec569e-8510-4b0e-8edc-8ff74143830b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:18.652703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec569e-8510-4b0e-8edc-8ff74143830b None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:18.655128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec569e-8510-4b0e-8edc-8ff74143830b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:18.655812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec569e-8510-4b0e-8edc-8ff74143830b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:18.660372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ec569e-8510-4b0e-8edc-8ff74143830b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:18.676947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:18.693755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf2795-ece6-4c7c-9e93-15f3b64f9230 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:18.698901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 372/753] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:18.700930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec569e-8510-4b0e-8edc-8ff74143830b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:18.737635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5912e-b6c6-4b32-a64f-7b350bcdc85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:18.738462 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5912e-b6c6-4b32-a64f-7b350bcdc85a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:18.738965 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5912e-b6c6-4b32-a64f-7b350bcdc85a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:18.739437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5912e-b6c6-4b32-a64f-7b350bcdc85a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:18.740485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5912e-b6c6-4b32-a64f-7b350bcdc85a None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=c3cf87f0515b460dad8c0ff5591c2685) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:18.740909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5912e-b6c6-4b32-a64f-7b350bcdc85a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:18.742613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5912e-b6c6-4b32-a64f-7b350bcdc85a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:18.750244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 382/754] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:18 2022] GET /identity/v3/role_assignments?user.id=c3cf87f0515b460dad8c0ff5591c2685&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:18.755854 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5912e-b6c6-4b32-a64f-7b350bcdc85a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:18.883352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:18.933822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec569e-8510-4b0e-8edc-8ff74143830b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:19.011379 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744179b9-3c86-4a28-ad6e-719d4208c65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:19.013113 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744179b9-3c86-4a28-ad6e-719d4208c65e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:19.013850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744179b9-3c86-4a28-ad6e-719d4208c65e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:19.014651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744179b9-3c86-4a28-ad6e-719d4208c65e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:19.037642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744179b9-3c86-4a28-ad6e-719d4208c65e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:19.040852 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744179b9-3c86-4a28-ad6e-719d4208c65e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:19.066753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5912e-b6c6-4b32-a64f-7b350bcdc85a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-744179b9-3c86-4a28-ad6e-719d4208c65e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:19.075531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:19.077237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:19.079593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:19.079593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:19.079593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:19.079593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:19.083403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 373/755] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:19.091866 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744179b9-3c86-4a28-ad6e-719d4208c65e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:19.159330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045149a-2cb1-4fe4-a662-e90d812345b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:19.160655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045149a-2cb1-4fe4-a662-e90d812345b1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:19.161229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045149a-2cb1-4fe4-a662-e90d812345b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:19.161711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045149a-2cb1-4fe4-a662-e90d812345b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:19.175948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045149a-2cb1-4fe4-a662-e90d812345b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:19.177806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045149a-2cb1-4fe4-a662-e90d812345b1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4045149a-2cb1-4fe4-a662-e90d812345b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:19.211299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:19.212188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:19.213264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:19.213264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:19.213264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:19.213264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:19.213264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:19.213264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:19.213264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:19.213264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:19.215692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 383/756] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 466 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:19.238137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045149a-2cb1-4fe4-a662-e90d812345b1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:19.239937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744179b9-3c86-4a28-ad6e-719d4208c65e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:19.279975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:19.292819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:19.302884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180873bc-8014-4da3-9ac1-69bbcaa50ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:19.304083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180873bc-8014-4da3-9ac1-69bbcaa50ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:19.305067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180873bc-8014-4da3-9ac1-69bbcaa50ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:19.305589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180873bc-8014-4da3-9ac1-69bbcaa50ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:19.307309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180873bc-8014-4da3-9ac1-69bbcaa50ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:19.307727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180873bc-8014-4da3-9ac1-69bbcaa50ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:19.311141 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180873bc-8014-4da3-9ac1-69bbcaa50ce3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:19.342991 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 374/757] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:19.350282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180873bc-8014-4da3-9ac1-69bbcaa50ce3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:19.479053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045149a-2cb1-4fe4-a662-e90d812345b1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:19.498191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180873bc-8014-4da3-9ac1-69bbcaa50ce3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:19.543183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da448c57-3556-47da-a1ce-85a49a9bd82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:19.544842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da448c57-3556-47da-a1ce-85a49a9bd82a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:19.545129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da448c57-3556-47da-a1ce-85a49a9bd82a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:19.545412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da448c57-3556-47da-a1ce-85a49a9bd82a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:19.557644 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da448c57-3556-47da-a1ce-85a49a9bd82a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:19.560095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da448c57-3556-47da-a1ce-85a49a9bd82a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:19.575369 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa82be5-e94e-42d3-a93f-ac47d178c05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:19.576459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82be5-e94e-42d3-a93f-ac47d178c05f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:19.576963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82be5-e94e-42d3-a93f-ac47d178c05f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:19.577466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82be5-e94e-42d3-a93f-ac47d178c05f None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:19.578474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa82be5-e94e-42d3-a93f-ac47d178c05f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:19.579676 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa82be5-e94e-42d3-a93f-ac47d178c05f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:19.581391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa82be5-e94e-42d3-a93f-ac47d178c05f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-da448c57-3556-47da-a1ce-85a49a9bd82a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:19.583063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:19.583983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:19.585345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:19.585345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:19.585345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:19.585345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:19.585345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:19.585345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:19.585345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:19.585345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:19.585345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:12:19.585345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:19.585345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 375/758] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Jul 20 08:12:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:19.592499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da448c57-3556-47da-a1ce-85a49a9bd82a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:19.611766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 384/759] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:19.618400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa82be5-e94e-42d3-a93f-ac47d178c05f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:19.684402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:19.742684 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:19.800614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa82be5-e94e-42d3-a93f-ac47d178c05f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:19.882360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b1e7d-ea3d-4cb7-9af9-7761f9e2ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:19.884137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b1e7d-ea3d-4cb7-9af9-7761f9e2ed86 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:19.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da448c57-3556-47da-a1ce-85a49a9bd82a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:19.887043 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b1e7d-ea3d-4cb7-9af9-7761f9e2ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:19.887817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b1e7d-ea3d-4cb7-9af9-7761f9e2ed86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:19.890935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b1e7d-ea3d-4cb7-9af9-7761f9e2ed86 None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=5284fc12641540c3a42f291138b7c069) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:19.894174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b1e7d-ea3d-4cb7-9af9-7761f9e2ed86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:19.897739 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b1e7d-ea3d-4cb7-9af9-7761f9e2ed86 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:19.909903 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 385/760] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:19 2022] GET /identity/v3/role_assignments?user.id=5284fc12641540c3a42f291138b7c069&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:19.945310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3593462e-6608-4ad8-93cb-681d0ab1deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:19.946036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593462e-6608-4ad8-93cb-681d0ab1deb0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:19.946323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593462e-6608-4ad8-93cb-681d0ab1deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:19.946668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593462e-6608-4ad8-93cb-681d0ab1deb0 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:19.947514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593462e-6608-4ad8-93cb-681d0ab1deb0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:19.947834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593462e-6608-4ad8-93cb-681d0ab1deb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:19.949477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3593462e-6608-4ad8-93cb-681d0ab1deb0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:19.971959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 376/761] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Jul 20 08:12:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:19.977815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b1e7d-ea3d-4cb7-9af9-7761f9e2ed86 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:20.062842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60c307-c7bd-45b4-97ef-60dde089e8b7 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:20.063315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60c307-c7bd-45b4-97ef-60dde089e8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:20.063478 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60c307-c7bd-45b4-97ef-60dde089e8b7 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:20.065230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 377/762] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:20.071285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf1562-3725-40c4-ad2a-5341301ada1e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:20.071693 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf1562-3725-40c4-ad2a-5341301ada1e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:20.071851 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf1562-3725-40c4-ad2a-5341301ada1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:20.098965 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-21cf1562-3725-40c4-ad2a-5341301ada1e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:20.100900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-21cf1562-3725-40c4-ad2a-5341301ada1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:20.253781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 378/763] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:20.270197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b1e7d-ea3d-4cb7-9af9-7761f9e2ed86 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:20.304292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf1562-3725-40c4-ad2a-5341301ada1e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:20.351400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d045a5-9e75-4e8c-97bf-9905a69190a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:20.352346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d045a5-9e75-4e8c-97bf-9905a69190a4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:20.352557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d045a5-9e75-4e8c-97bf-9905a69190a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:20.352929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d045a5-9e75-4e8c-97bf-9905a69190a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:20.359779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:20.383160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:20.393802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d045a5-9e75-4e8c-97bf-9905a69190a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:20.395629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d045a5-9e75-4e8c-97bf-9905a69190a4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-30d045a5-9e75-4e8c-97bf-9905a69190a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:20.432979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:20.434231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:20.435292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:20.435292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:20.435292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:20.435292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:20.435292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 386/764] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:20.443742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d045a5-9e75-4e8c-97bf-9905a69190a4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:20.475096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf1562-3725-40c4-ad2a-5341301ada1e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:20.521446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f4a77-2fcc-4444-acab-256c0b594a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:20.522712 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4a77-2fcc-4444-acab-256c0b594a23 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:20.523314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4a77-2fcc-4444-acab-256c0b594a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:20.523790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f4a77-2fcc-4444-acab-256c0b594a23 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:20.524739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f4a77-2fcc-4444-acab-256c0b594a23 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:20.526480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f4a77-2fcc-4444-acab-256c0b594a23 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-956f4a77-2fcc-4444-acab-256c0b594a23 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:20.541217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:20.542204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 08:12:20.543175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:20.545768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 379/765] 10.222.0.64 () {56 vars in 1129 bytes} [Wed Jul 20 08:12:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:20.547422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdea9dc-7877-4a27-be66-0031b9fffc57 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:20.547930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdea9dc-7877-4a27-be66-0031b9fffc57 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:20.548390 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdea9dc-7877-4a27-be66-0031b9fffc57 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:20.550540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 380/766] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:20.562209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04febf-c05c-45b0-86b6-82b369dd2533 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:20.562755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04febf-c05c-45b0-86b6-82b369dd2533 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:20.564011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04febf-c05c-45b0-86b6-82b369dd2533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:20.586468 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5a04febf-c05c-45b0-86b6-82b369dd2533 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:20.588128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5a04febf-c05c-45b0-86b6-82b369dd2533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:20.644507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d045a5-9e75-4e8c-97bf-9905a69190a4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:20.698995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 381/767] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:20.700707 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04febf-c05c-45b0-86b6-82b369dd2533 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:20.725022 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3f479-7fd0-4cb6-982d-1090cf86f865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:20.726181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3f479-7fd0-4cb6-982d-1090cf86f865 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:20.726717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3f479-7fd0-4cb6-982d-1090cf86f865 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:20.727225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3f479-7fd0-4cb6-982d-1090cf86f865 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:20.728802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3f479-7fd0-4cb6-982d-1090cf86f865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:20.729435 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3f479-7fd0-4cb6-982d-1090cf86f865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:20.739187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:20.739473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3f479-7fd0-4cb6-982d-1090cf86f865 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:20.765638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 387/768] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:20.774821 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3f479-7fd0-4cb6-982d-1090cf86f865 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:20.831157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04febf-c05c-45b0-86b6-82b369dd2533 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:20.870287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bdce1-0342-4f5b-876f-2853c69d2f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:20.871404 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bdce1-0342-4f5b-876f-2853c69d2f43 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:20.871838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bdce1-0342-4f5b-876f-2853c69d2f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:20.871952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:20.872360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bdce1-0342-4f5b-876f-2853c69d2f43 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:20.873275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bdce1-0342-4f5b-876f-2853c69d2f43 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:20.873769 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bdce1-0342-4f5b-876f-2853c69d2f43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:20.875740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bdce1-0342-4f5b-876f-2853c69d2f43 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:20.886996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 382/769] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Jul 20 08:12:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:20.888163 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bdce1-0342-4f5b-876f-2853c69d2f43 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:20.974500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3f479-7fd0-4cb6-982d-1090cf86f865 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:20.997777 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:21.007589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bdce1-0342-4f5b-876f-2853c69d2f43 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:21.044896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07882aa-a153-40f0-ba87-ac14f1a79362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:21.046438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07882aa-a153-40f0-ba87-ac14f1a79362 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:21.047107 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07882aa-a153-40f0-ba87-ac14f1a79362 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:21.048273 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07882aa-a153-40f0-ba87-ac14f1a79362 None admin] PATH_INFO: `/v3/services/network` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:21.049378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f219b62-bb75-48ab-80e2-7e6b601e92c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:21.049794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07882aa-a153-40f0-ba87-ac14f1a79362 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:21.050524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f219b62-bb75-48ab-80e2-7e6b601e92c1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:21.050982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f219b62-bb75-48ab-80e2-7e6b601e92c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:21.051417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f219b62-bb75-48ab-80e2-7e6b601e92c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:21.052806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07882aa-a153-40f0-ba87-ac14f1a79362 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:21.059302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f219b62-bb75-48ab-80e2-7e6b601e92c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:21.060943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f219b62-bb75-48ab-80e2-7e6b601e92c1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-5f219b62-bb75-48ab-80e2-7e6b601e92c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:21.076275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:21.077302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:21.078308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:21.078308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:21.078308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:21.078308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:21.078308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:21.078308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:21.078308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:21.078308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a07882aa-a153-40f0-ba87-ac14f1a79362 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:21.079299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:21.080322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 08:12:21.081322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:21.083811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 383/770] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:21.084763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f219b62-bb75-48ab-80e2-7e6b601e92c1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:21.097042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 388/771] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:20 2022] GET /identity/v3/services/network => generated 88 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:21.097042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07882aa-a153-40f0-ba87-ac14f1a79362 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:21.238131 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07882aa-a153-40f0-ba87-ac14f1a79362 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:21.258040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:21.290941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b73614-aa9c-4c33-a1ac-a10eee51e441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:21.292533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b73614-aa9c-4c33-a1ac-a10eee51e441 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:21.293037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b73614-aa9c-4c33-a1ac-a10eee51e441 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:21.293652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b73614-aa9c-4c33-a1ac-a10eee51e441 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:21.295181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b73614-aa9c-4c33-a1ac-a10eee51e441 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:21.295725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b73614-aa9c-4c33-a1ac-a10eee51e441 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:21.297441 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b73614-aa9c-4c33-a1ac-a10eee51e441 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:21.304649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f219b62-bb75-48ab-80e2-7e6b601e92c1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:21.314021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 389/772] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:21.315236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b73614-aa9c-4c33-a1ac-a10eee51e441 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:21.357824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efaa11-cc0a-4c18-8cc1-bdc224e50915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:21.358928 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efaa11-cc0a-4c18-8cc1-bdc224e50915 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:21.359445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efaa11-cc0a-4c18-8cc1-bdc224e50915 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:21.359925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42efaa11-cc0a-4c18-8cc1-bdc224e50915 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:21.361246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efaa11-cc0a-4c18-8cc1-bdc224e50915 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:21.361828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efaa11-cc0a-4c18-8cc1-bdc224e50915 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:21.363458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42efaa11-cc0a-4c18-8cc1-bdc224e50915 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:21.370521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 384/773] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Jul 20 08:12:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:21.371826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efaa11-cc0a-4c18-8cc1-bdc224e50915 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:21.402174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:21.472062 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b73614-aa9c-4c33-a1ac-a10eee51e441 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:21.498249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42efaa11-cc0a-4c18-8cc1-bdc224e50915 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:21.522925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7558a6-4d93-48df-afaa-7753a12a7b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:21.523722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7558a6-4d93-48df-afaa-7753a12a7b31 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:21.524017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7558a6-4d93-48df-afaa-7753a12a7b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:21.524302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7558a6-4d93-48df-afaa-7753a12a7b31 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:21.525691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7558a6-4d93-48df-afaa-7753a12a7b31 None admin] RBAC: Adding query filter params (name=network) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:21.526148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7558a6-4d93-48df-afaa-7753a12a7b31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:21.527922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7558a6-4d93-48df-afaa-7753a12a7b31 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:21.540715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 390/774] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:21.540715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7558a6-4d93-48df-afaa-7753a12a7b31 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:21.549016 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e8022-0563-4799-bc24-ae3c795900bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:21.550401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e8022-0563-4799-bc24-ae3c795900bd None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:21.550980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e8022-0563-4799-bc24-ae3c795900bd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:21.551456 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e8022-0563-4799-bc24-ae3c795900bd None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/84f0da8709624834976d0662aeb02ea6/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:21.553907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e8022-0563-4799-bc24-ae3c795900bd None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=84f0da8709624834976d0662aeb02ea6, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:21.555676 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5e8022-0563-4799-bc24-ae3c795900bd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:21.576849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 385/775] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:12:21 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/84f0da8709624834976d0662aeb02ea6/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:12:21.578021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e8022-0563-4799-bc24-ae3c795900bd None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:21.617646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:21.755159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:21.785160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e8022-0563-4799-bc24-ae3c795900bd None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:21.838025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4b06-d26a-4163-b618-4b61e737edb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:21.839205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4b06-d26a-4163-b618-4b61e737edb8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:21.839697 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4b06-d26a-4163-b618-4b61e737edb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:21.840205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa4b06-d26a-4163-b618-4b61e737edb8 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:21.841170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4b06-d26a-4163-b618-4b61e737edb8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:21.841732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4b06-d26a-4163-b618-4b61e737edb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:21.843974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa4b06-d26a-4163-b618-4b61e737edb8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:21.844838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7558a6-4d93-48df-afaa-7753a12a7b31 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:21.851454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 386/776] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:21.859300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4b06-d26a-4163-b618-4b61e737edb8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:21.900542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98188d8-cbe9-4536-99c7-411d17282c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:21.901139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98188d8-cbe9-4536-99c7-411d17282c01 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:21.901429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98188d8-cbe9-4536-99c7-411d17282c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:21.901712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98188d8-cbe9-4536-99c7-411d17282c01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:21.902936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98188d8-cbe9-4536-99c7-411d17282c01 None admin] RBAC: Adding query filter params (interface=public, service_id=604bf0d0d41e462aa69b0e762cf6fed4, region_id=RegionOne) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:21.902936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98188d8-cbe9-4536-99c7-411d17282c01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:21.904531 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98188d8-cbe9-4536-99c7-411d17282c01 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:21.914976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 391/777] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Jul 20 08:12:21 2022] GET /identity/v3/endpoints?service_id=604bf0d0d41e462aa69b0e762cf6fed4&interface=public®ion_id=RegionOne => generated 194 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:21.916391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37b7f2-203d-4bda-af18-79d2a94b8404 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:21.916681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37b7f2-203d-4bda-af18-79d2a94b8404 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:21.916963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37b7f2-203d-4bda-af18-79d2a94b8404 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:21.918611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 392/778] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:21.923710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37b7f2-203d-4bda-af18-79d2a94b8404 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:21.994474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa4b06-d26a-4163-b618-4b61e737edb8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:22.035364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b4c3a9-8fe7-4a52-bdac-8c4212397f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:22.036050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c3a9-8fe7-4a52-bdac-8c4212397f15 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:22.036338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c3a9-8fe7-4a52-bdac-8c4212397f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:22.036622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b4c3a9-8fe7-4a52-bdac-8c4212397f15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:22.037720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4c3a9-8fe7-4a52-bdac-8c4212397f15 None admin] RBAC: Adding query filter params (interface=public, service_id=e0b33ce4da7044dbbe6d1b4cc903b409, region_id=RegionOne) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:22.038082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4c3a9-8fe7-4a52-bdac-8c4212397f15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:22.039800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b4c3a9-8fe7-4a52-bdac-8c4212397f15 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:22.050649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 387/779] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Jul 20 08:12:21 2022] GET /identity/v3/endpoints?service_id=e0b33ce4da7044dbbe6d1b4cc903b409&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:22.052207 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d39916-d489-4912-9354-38e6ed7a0227 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:22.052549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d39916-d489-4912-9354-38e6ed7a0227 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:22.052862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d39916-d489-4912-9354-38e6ed7a0227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:22.074414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c4d39916-d489-4912-9354-38e6ed7a0227 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:22.076081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c4d39916-d489-4912-9354-38e6ed7a0227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:22.083979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37b7f2-203d-4bda-af18-79d2a94b8404 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:22.085340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:22.086441 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:22.133851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046894c-404d-4413-a5c5-08b06e40a179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:22.134566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046894c-404d-4413-a5c5-08b06e40a179 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:22.134887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046894c-404d-4413-a5c5-08b06e40a179 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:22.135172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046894c-404d-4413-a5c5-08b06e40a179 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:22.135984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046894c-404d-4413-a5c5-08b06e40a179 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:22.138434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046894c-404d-4413-a5c5-08b06e40a179 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:22.146882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 393/780] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Jul 20 08:12:21 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:22.148148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046894c-404d-4413-a5c5-08b06e40a179 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:22.169324 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 388/781] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:22.197084 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d39916-d489-4912-9354-38e6ed7a0227 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:22.347209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046894c-404d-4413-a5c5-08b06e40a179 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:22.378137 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d39916-d489-4912-9354-38e6ed7a0227 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:22.391104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f46019-1343-425a-8c8d-1057526edcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:22.392975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f46019-1343-425a-8c8d-1057526edcae None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:22.393314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f46019-1343-425a-8c8d-1057526edcae None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:22.393589 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f46019-1343-425a-8c8d-1057526edcae None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:22.394303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f46019-1343-425a-8c8d-1057526edcae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:22.395828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f46019-1343-425a-8c8d-1057526edcae None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:22.402322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 394/782] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Jul 20 08:12:22 2022] GET /identity/v3/services => generated 772 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:22.403555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548d99-3a0b-4f79-a2a8-7f5d7096964a None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:22.403883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548d99-3a0b-4f79-a2a8-7f5d7096964a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:22.404184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548d99-3a0b-4f79-a2a8-7f5d7096964a None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:22.405962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 395/783] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:22.414237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb841a-d050-4b9d-92cd-f144247f93c2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:22.414635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb841a-d050-4b9d-92cd-f144247f93c2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:22.414828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb841a-d050-4b9d-92cd-f144247f93c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:22.447408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-34fb841a-d050-4b9d-92cd-f144247f93c2 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:22.450404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-34fb841a-d050-4b9d-92cd-f144247f93c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:22.487116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:22.565351 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 396/784] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:22.569194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34ddc7-9a2d-45f8-8a4e-ebec6abf375e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:22.570071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34ddc7-9a2d-45f8-8a4e-ebec6abf375e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:22.570357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34ddc7-9a2d-45f8-8a4e-ebec6abf375e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:22.570675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34ddc7-9a2d-45f8-8a4e-ebec6abf375e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:22.581104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:22.587433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34ddc7-9a2d-45f8-8a4e-ebec6abf375e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-0d34ddc7-9a2d-45f8-8a4e-ebec6abf375e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:22.597011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:22.598102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:22.598938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:22.602421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 389/785] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:12:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:22.611641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb841a-d050-4b9d-92cd-f144247f93c2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:22.621418 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34ddc7-9a2d-45f8-8a4e-ebec6abf375e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:22.745266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34ddc7-9a2d-45f8-8a4e-ebec6abf375e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:22.746101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb841a-d050-4b9d-92cd-f144247f93c2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:22.788102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a47b2-e135-4655-aa76-e16f779b5489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:22.788902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a47b2-e135-4655-aa76-e16f779b5489 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:22.789188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a47b2-e135-4655-aa76-e16f779b5489 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:22.789467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a47b2-e135-4655-aa76-e16f779b5489 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:22.792104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef204b74-45ed-4297-a668-043d0c3d89c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:22.793129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef204b74-45ed-4297-a668-043d0c3d89c3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:22.793592 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef204b74-45ed-4297-a668-043d0c3d89c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:22.794041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef204b74-45ed-4297-a668-043d0c3d89c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:22.795082 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef204b74-45ed-4297-a668-043d0c3d89c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:22.795598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef204b74-45ed-4297-a668-043d0c3d89c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:22.797171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef204b74-45ed-4297-a668-043d0c3d89c3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:22.803117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a47b2-e135-4655-aa76-e16f779b5489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:22.808178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 397/786] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:12:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-5c2a47b2-e135-4655-aa76-e16f779b5489 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:22.812409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:22.814068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:22.815165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:22.816234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 390/787] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:12:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:22.817356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef204b74-45ed-4297-a668-043d0c3d89c3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:22.822903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a47b2-e135-4655-aa76-e16f779b5489 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:22.856698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:22.892160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:22.942983 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef204b74-45ed-4297-a668-043d0c3d89c3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:22.958487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a47b2-e135-4655-aa76-e16f779b5489 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:22.983671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472dc450-91c0-4c2d-a3f4-543cbb0f90dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:22.984792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc450-91c0-4c2d-a3f4-543cbb0f90dc None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:22.986140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc450-91c0-4c2d-a3f4-543cbb0f90dc None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:22.986893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472dc450-91c0-4c2d-a3f4-543cbb0f90dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:22.999612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaff61d-5b22-4561-b6ae-7ec19f32ce4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:23.000170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dc450-91c0-4c2d-a3f4-543cbb0f90dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:23.000317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaff61d-5b22-4561-b6ae-7ec19f32ce4b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:23.000738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaff61d-5b22-4561-b6ae-7ec19f32ce4b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:23.001535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaff61d-5b22-4561-b6ae-7ec19f32ce4b None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:23.002706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaff61d-5b22-4561-b6ae-7ec19f32ce4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:23.003023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaff61d-5b22-4561-b6ae-7ec19f32ce4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:23.006205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaff61d-5b22-4561-b6ae-7ec19f32ce4b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:23.014624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 391/788] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:12:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:23.016084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472dc450-91c0-4c2d-a3f4-543cbb0f90dc None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:23.021295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaff61d-5b22-4561-b6ae-7ec19f32ce4b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-472dc450-91c0-4c2d-a3f4-543cbb0f90dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:23.056276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:23.057214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:23.058097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:23.058097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:23.058097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:23.058097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:23.060284 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 398/789] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:23.072619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472dc450-91c0-4c2d-a3f4-543cbb0f90dc None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:23.252237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaff61d-5b22-4561-b6ae-7ec19f32ce4b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:23.257956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:23.274414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:23.305753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88071496-dd76-48fa-ae17-192eb45dd034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:23.306562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071496-dd76-48fa-ae17-192eb45dd034 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:23.306929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071496-dd76-48fa-ae17-192eb45dd034 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:23.307207 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071496-dd76-48fa-ae17-192eb45dd034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:23.318935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472dc450-91c0-4c2d-a3f4-543cbb0f90dc None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:23.335551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88071496-dd76-48fa-ae17-192eb45dd034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:23.337031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88071496-dd76-48fa-ae17-192eb45dd034 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-88071496-dd76-48fa-ae17-192eb45dd034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:23.370190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:23.371549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:23.372892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:23.372892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:23.372892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:23.372892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:23.372892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 392/790] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:23.377680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5a5e-0a9a-4efd-b9f9-c679f5332b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:23.378267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88071496-dd76-48fa-ae17-192eb45dd034 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:23.379345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5a5e-0a9a-4efd-b9f9-c679f5332b0c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:23.379879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5a5e-0a9a-4efd-b9f9-c679f5332b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:23.380396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5a5e-0a9a-4efd-b9f9-c679f5332b0c None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:23.381452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b5a5e-0a9a-4efd-b9f9-c679f5332b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:23.381972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b5a5e-0a9a-4efd-b9f9-c679f5332b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:23.383631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b5a5e-0a9a-4efd-b9f9-c679f5332b0c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:23.399231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 399/791] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:23.405313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5a5e-0a9a-4efd-b9f9-c679f5332b0c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:23.579775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88071496-dd76-48fa-ae17-192eb45dd034 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:23.635510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5a5e-0a9a-4efd-b9f9-c679f5332b0c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:23.653316 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee9d1c-5cee-4048-89b2-79c2a5877289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:23.654482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9d1c-5cee-4048-89b2-79c2a5877289 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:23.655027 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9d1c-5cee-4048-89b2-79c2a5877289 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:23.655530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee9d1c-5cee-4048-89b2-79c2a5877289 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:23.656753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee9d1c-5cee-4048-89b2-79c2a5877289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:23.657605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee9d1c-5cee-4048-89b2-79c2a5877289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:23.659913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee9d1c-5cee-4048-89b2-79c2a5877289 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:23.690784 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 393/792] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:23.699654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee9d1c-5cee-4048-89b2-79c2a5877289 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:23.703039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29ef9b-3023-450f-9a04-294fe1490db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:23.705411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29ef9b-3023-450f-9a04-294fe1490db4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:23.705752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29ef9b-3023-450f-9a04-294fe1490db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:23.706033 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29ef9b-3023-450f-9a04-294fe1490db4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:23.715634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29ef9b-3023-450f-9a04-294fe1490db4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:23.717106 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29ef9b-3023-450f-9a04-294fe1490db4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:23.727047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-cb29ef9b-3023-450f-9a04-294fe1490db4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:23.740792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:23.742870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:23.744743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:23.744743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:23.744743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:23.744743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:23.744743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:23.744743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:23.744743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:23.744743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:23.744743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:12:23.744743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:23.744743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 400/793] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:12:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:23.750088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29ef9b-3023-450f-9a04-294fe1490db4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:23.816943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:24.014725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee9d1c-5cee-4048-89b2-79c2a5877289 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:24.018356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29ef9b-3023-450f-9a04-294fe1490db4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:24.085660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b410f2-7943-4d2e-b0b8-b8340af54b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:24.086344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b410f2-7943-4d2e-b0b8-b8340af54b44 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:24.086647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b410f2-7943-4d2e-b0b8-b8340af54b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:24.086934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b410f2-7943-4d2e-b0b8-b8340af54b44 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:24.087771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b410f2-7943-4d2e-b0b8-b8340af54b44 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:24.088083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b410f2-7943-4d2e-b0b8-b8340af54b44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:24.089548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b410f2-7943-4d2e-b0b8-b8340af54b44 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:24.095183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993b7d1-ce8b-4e23-b13d-68586fe39331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:24.096010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993b7d1-ce8b-4e23-b13d-68586fe39331 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:24.096291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993b7d1-ce8b-4e23-b13d-68586fe39331 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:24.096571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993b7d1-ce8b-4e23-b13d-68586fe39331 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:24.111227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993b7d1-ce8b-4e23-b13d-68586fe39331 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:24.119571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993b7d1-ce8b-4e23-b13d-68586fe39331 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:24.133415 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 401/794] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Jul 20 08:12:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:24.138909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b410f2-7943-4d2e-b0b8-b8340af54b44 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-f993b7d1-ce8b-4e23-b13d-68586fe39331 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:24.148812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:24.150858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:24.152460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:24.152460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:24.152460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:24.152460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:24.152460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:24.152460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:24.152460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:24.152460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:24.152460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:24.152460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:24.153583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 394/795] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Jul 20 08:12:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:24.173392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993b7d1-ce8b-4e23-b13d-68586fe39331 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:24.266076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:24.359776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:24.373894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993b7d1-ce8b-4e23-b13d-68586fe39331 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:24.419886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b410f2-7943-4d2e-b0b8-b8340af54b44 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:24.421657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93622ed9-459b-4900-90af-898bb1a75b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:24.423559 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93622ed9-459b-4900-90af-898bb1a75b35 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:24.424187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93622ed9-459b-4900-90af-898bb1a75b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:24.424678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93622ed9-459b-4900-90af-898bb1a75b35 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:24.425828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93622ed9-459b-4900-90af-898bb1a75b35 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:24.426699 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93622ed9-459b-4900-90af-898bb1a75b35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:24.431311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93622ed9-459b-4900-90af-898bb1a75b35 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:24.471292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 395/796] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Jul 20 08:12:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:24.472905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147193c5-88c0-4667-abc2-542454b7dc74 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:24.473198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147193c5-88c0-4667-abc2-542454b7dc74 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:24.473474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147193c5-88c0-4667-abc2-542454b7dc74 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:24.475062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 396/797] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:24.478237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147193c5-88c0-4667-abc2-542454b7dc74 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:24.511179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f20285-0e4a-43ec-9565-fa65abc3947b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:24.513934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20285-0e4a-43ec-9565-fa65abc3947b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:24.514755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20285-0e4a-43ec-9565-fa65abc3947b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:24.517322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20285-0e4a-43ec-9565-fa65abc3947b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:24.539572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f20285-0e4a-43ec-9565-fa65abc3947b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:24.543316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f20285-0e4a-43ec-9565-fa65abc3947b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-b4f20285-0e4a-43ec-9565-fa65abc3947b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:24.569811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:24.570979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:24.572349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:24.572349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:24.572349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:24.572349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:24.579946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 402/798] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:24.582246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca26b0-8201-434a-ae26-a40ae58403a3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:24.582615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca26b0-8201-434a-ae26-a40ae58403a3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:24.583017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca26b0-8201-434a-ae26-a40ae58403a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:24.616359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9aca26b0-8201-434a-ae26-a40ae58403a3 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:24.618245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9aca26b0-8201-434a-ae26-a40ae58403a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:24.667438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147193c5-88c0-4667-abc2-542454b7dc74 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:24.712355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 403/799] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:24.713792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca26b0-8201-434a-ae26-a40ae58403a3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:24.725443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7142d-140f-4b2f-9e42-237168f5492c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:24.727483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7142d-140f-4b2f-9e42-237168f5492c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:24.728008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7142d-140f-4b2f-9e42-237168f5492c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:24.728494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7142d-140f-4b2f-9e42-237168f5492c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:24.744503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7142d-140f-4b2f-9e42-237168f5492c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:24.746925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7142d-140f-4b2f-9e42-237168f5492c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:24.763642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-acc7142d-140f-4b2f-9e42-237168f5492c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:24.772059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:24.772966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:24.773850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:24.773850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:24.773850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:24.773850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:24.773850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 397/800] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:24.774555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7142d-140f-4b2f-9e42-237168f5492c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:24.782930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:24.875261 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca26b0-8201-434a-ae26-a40ae58403a3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:24.908832 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7142d-140f-4b2f-9e42-237168f5492c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:24.925226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ba4f4-0b67-4f16-92c9-d4ff16f393a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:24.925925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ba4f4-0b67-4f16-92c9-d4ff16f393a1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:24.926227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ba4f4-0b67-4f16-92c9-d4ff16f393a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:24.926478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ba4f4-0b67-4f16-92c9-d4ff16f393a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:24.927266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ba4f4-0b67-4f16-92c9-d4ff16f393a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:24.927575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ba4f4-0b67-4f16-92c9-d4ff16f393a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:24.929010 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ba4f4-0b67-4f16-92c9-d4ff16f393a1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:24.950296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 404/801] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:24.950296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ba4f4-0b67-4f16-92c9-d4ff16f393a1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:24.962276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb5763b-3acb-4ffe-a732-84fe7e50c33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:24.963200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5763b-3acb-4ffe-a732-84fe7e50c33c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:24.963494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5763b-3acb-4ffe-a732-84fe7e50c33c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:24.963775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb5763b-3acb-4ffe-a732-84fe7e50c33c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:24.975298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb5763b-3acb-4ffe-a732-84fe7e50c33c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-1fb5763b-3acb-4ffe-a732-84fe7e50c33c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:24.983194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:24.984276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:24.985578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 398/802] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:12:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:24.986685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a70cd2-1b43-44a1-8138-fd05ee0a6eb3 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:24.986685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a70cd2-1b43-44a1-8138-fd05ee0a6eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:24.986685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a70cd2-1b43-44a1-8138-fd05ee0a6eb3 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:24.992446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 399/803] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:24.992446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a70cd2-1b43-44a1-8138-fd05ee0a6eb3 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:25.079488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:25.092950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:25.099518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ba4f4-0b67-4f16-92c9-d4ff16f393a1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:25.144069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a70cd2-1b43-44a1-8138-fd05ee0a6eb3 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:25.147005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ddc9e-1a41-4a09-97f5-bb6bbfe217dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:25.148009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ddc9e-1a41-4a09-97f5-bb6bbfe217dc None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:25.148525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ddc9e-1a41-4a09-97f5-bb6bbfe217dc None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:25.149002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ddc9e-1a41-4a09-97f5-bb6bbfe217dc None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:25.150478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ddc9e-1a41-4a09-97f5-bb6bbfe217dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:25.151085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ddc9e-1a41-4a09-97f5-bb6bbfe217dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:25.153786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ddc9e-1a41-4a09-97f5-bb6bbfe217dc None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:25.168622 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 405/804] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:25.169480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ddc9e-1a41-4a09-97f5-bb6bbfe217dc None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:25.197776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b3eca-15ef-492d-87ac-737211414279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:25.198775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b3eca-15ef-492d-87ac-737211414279 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:25.199138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b3eca-15ef-492d-87ac-737211414279 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:25.199465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b3eca-15ef-492d-87ac-737211414279 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:25.208052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b3eca-15ef-492d-87ac-737211414279 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:25.209621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b3eca-15ef-492d-87ac-737211414279 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-4b7b3eca-15ef-492d-87ac-737211414279 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:25.226198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:25.227384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:25.228260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:25.228260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:25.228260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:25.228260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:25.228260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:25.228260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:25.228260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:25.228260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:25.228260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 400/805] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:25.228753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfd66c-fbc1-42cf-bf2b-0aab414f1693 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:25.228966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfd66c-fbc1-42cf-bf2b-0aab414f1693 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:25.229253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfd66c-fbc1-42cf-bf2b-0aab414f1693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:25.248816 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e0bfd66c-fbc1-42cf-bf2b-0aab414f1693 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:25.250059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e0bfd66c-fbc1-42cf-bf2b-0aab414f1693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:25.300211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ddc9e-1a41-4a09-97f5-bb6bbfe217dc None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:25.327650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 401/806] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:25.328842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfd66c-fbc1-42cf-bf2b-0aab414f1693 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:25.338905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d8688-4f0d-438e-a234-61f747b265a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:25.339611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d8688-4f0d-438e-a234-61f747b265a8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:25.339905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d8688-4f0d-438e-a234-61f747b265a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:25.340186 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d8688-4f0d-438e-a234-61f747b265a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:25.340998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d8688-4f0d-438e-a234-61f747b265a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:25.341314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d8688-4f0d-438e-a234-61f747b265a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:25.342928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d8688-4f0d-438e-a234-61f747b265a8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:25.353082 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 406/807] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:12:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:25.353978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d8688-4f0d-438e-a234-61f747b265a8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:25.374829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:25.399774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:25.459490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfd66c-fbc1-42cf-bf2b-0aab414f1693 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:25.469176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d8688-4f0d-438e-a234-61f747b265a8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:25.497363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce00b-80e6-4278-ac40-89b52cab9489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:25.498174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce00b-80e6-4278-ac40-89b52cab9489 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:25.498461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce00b-80e6-4278-ac40-89b52cab9489 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:25.498771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce00b-80e6-4278-ac40-89b52cab9489 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:25.506166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dce00b-80e6-4278-ac40-89b52cab9489 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:25.507672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dce00b-80e6-4278-ac40-89b52cab9489 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:25.508611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5210fe-9b45-42b2-a3d5-ad5bb5723650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:25.509302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5210fe-9b45-42b2-a3d5-ad5bb5723650 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:25.509771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5210fe-9b45-42b2-a3d5-ad5bb5723650 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:25.510052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5210fe-9b45-42b2-a3d5-ad5bb5723650 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:25.511277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5210fe-9b45-42b2-a3d5-ad5bb5723650 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:25.511563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5210fe-9b45-42b2-a3d5-ad5bb5723650 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:25.513030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5210fe-9b45-42b2-a3d5-ad5bb5723650 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-59dce00b-80e6-4278-ac40-89b52cab9489 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:25.522533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:25.523742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:25.524680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:25.524680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:25.524680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:25.524680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:25.524680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:25.524680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:25.524680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:25.524680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:25.524680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 402/808] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:25.525391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c035c3-01a8-4ba4-8273-c6855c8efa9d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:25.525391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c035c3-01a8-4ba4-8273-c6855c8efa9d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:25.525664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c035c3-01a8-4ba4-8273-c6855c8efa9d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:25.527189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 403/809] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:25.527683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c035c3-01a8-4ba4-8273-c6855c8efa9d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:25.537028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 407/810] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:25.538165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5210fe-9b45-42b2-a3d5-ad5bb5723650 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:25.642068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:25.666643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c035c3-01a8-4ba4-8273-c6855c8efa9d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:25.672126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5210fe-9b45-42b2-a3d5-ad5bb5723650 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:25.683249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:25.710617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ca1d7-3816-474c-839c-b5fd74445f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:25.711460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca1d7-3816-474c-839c-b5fd74445f98 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:25.711740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca1d7-3816-474c-839c-b5fd74445f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:25.712009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca1d7-3816-474c-839c-b5fd74445f98 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:25.712682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ca1d7-3816-474c-839c-b5fd74445f98 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:25.714062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ca1d7-3816-474c-839c-b5fd74445f98 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:25.720119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac5fc8-0607-4990-a680-0f57bddabe76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:25.720951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac5fc8-0607-4990-a680-0f57bddabe76 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:25.721248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac5fc8-0607-4990-a680-0f57bddabe76 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:25.721494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac5fc8-0607-4990-a680-0f57bddabe76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-1a9ca1d7-3816-474c-839c-b5fd74445f98 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:25.731058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:25.732499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:25.734338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 404/811] 10.222.0.64 () {56 vars in 1129 bytes} [Wed Jul 20 08:12:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:25.736903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ca1d7-3816-474c-839c-b5fd74445f98 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:25.743333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac5fc8-0607-4990-a680-0f57bddabe76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:25.744307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac5fc8-0607-4990-a680-0f57bddabe76 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-46ac5fc8-0607-4990-a680-0f57bddabe76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:25.768473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:25.769507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:25.770731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:25.770731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:25.770731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:25.770731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:25.770731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 408/812] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:25.771194 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0d2eb-ccb1-4cb2-992a-0952ee71bc5d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:25.771354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0d2eb-ccb1-4cb2-992a-0952ee71bc5d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:25.771637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0d2eb-ccb1-4cb2-992a-0952ee71bc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:25.789891 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6df0d2eb-ccb1-4cb2-992a-0952ee71bc5d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:25.791165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6df0d2eb-ccb1-4cb2-992a-0952ee71bc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:25.866357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ca1d7-3816-474c-839c-b5fd74445f98 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:25.880501 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 409/813] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:25.881504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0d2eb-ccb1-4cb2-992a-0952ee71bc5d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:25.918316 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfa565-7954-4590-96b8-36495b377d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:25.918947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa565-7954-4590-96b8-36495b377d9d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:25.919237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa565-7954-4590-96b8-36495b377d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:25.919515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfa565-7954-4590-96b8-36495b377d9d None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:25.920318 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfa565-7954-4590-96b8-36495b377d9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:25.920631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfa565-7954-4590-96b8-36495b377d9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:25.922118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfa565-7954-4590-96b8-36495b377d9d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:25.943778 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 405/814] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:25.943778 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfa565-7954-4590-96b8-36495b377d9d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:25.966452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:25.978244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:26.023006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0d2eb-ccb1-4cb2-992a-0952ee71bc5d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:26.062486 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfa565-7954-4590-96b8-36495b377d9d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:26.070003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfdf6f2-b595-43b5-936d-cfa7a68a740e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:26.070956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdf6f2-b595-43b5-936d-cfa7a68a740e None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:26.071234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdf6f2-b595-43b5-936d-cfa7a68a740e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:26.071512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdf6f2-b595-43b5-936d-cfa7a68a740e None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/c3cf87f0515b460dad8c0ff5591c2685/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:26.073618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfdf6f2-b595-43b5-936d-cfa7a68a740e None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=c3cf87f0515b460dad8c0ff5591c2685, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:26.075143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfdf6f2-b595-43b5-936d-cfa7a68a740e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:26.090208 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 410/815] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:12:25 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/c3cf87f0515b460dad8c0ff5591c2685/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:12:26.091216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfdf6f2-b595-43b5-936d-cfa7a68a740e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:26.118219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c2096-234f-4547-8298-c94e1dd0f920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:26.119059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c2096-234f-4547-8298-c94e1dd0f920 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:26.119422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c2096-234f-4547-8298-c94e1dd0f920 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:26.119780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c2096-234f-4547-8298-c94e1dd0f920 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:26.120615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c2096-234f-4547-8298-c94e1dd0f920 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:26.121010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c2096-234f-4547-8298-c94e1dd0f920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:26.122506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c2096-234f-4547-8298-c94e1dd0f920 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:26.128279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 406/816] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Jul 20 08:12:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:26.128994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c2096-234f-4547-8298-c94e1dd0f920 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:26.237416 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfdf6f2-b595-43b5-936d-cfa7a68a740e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:26.258618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:26.264872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:26.268112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c2096-234f-4547-8298-c94e1dd0f920 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:26.287168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062075ca-ccb2-4e9a-9bb6-40e6eb9953da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:26.287856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062075ca-ccb2-4e9a-9bb6-40e6eb9953da None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:26.288196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062075ca-ccb2-4e9a-9bb6-40e6eb9953da None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:26.288482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062075ca-ccb2-4e9a-9bb6-40e6eb9953da None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:26.289236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062075ca-ccb2-4e9a-9bb6-40e6eb9953da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:26.289548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062075ca-ccb2-4e9a-9bb6-40e6eb9953da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:26.291073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062075ca-ccb2-4e9a-9bb6-40e6eb9953da None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:26.304294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 411/817] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:26.304552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062075ca-ccb2-4e9a-9bb6-40e6eb9953da None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:26.319778 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a8dd4-5cf1-467e-a0ed-081b14c6a799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:26.320913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a8dd4-5cf1-467e-a0ed-081b14c6a799 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:26.321233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a8dd4-5cf1-467e-a0ed-081b14c6a799 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:26.321581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a8dd4-5cf1-467e-a0ed-081b14c6a799 None admin] PATH_INFO: `/v3/services/network` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:26.322471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a8dd4-5cf1-467e-a0ed-081b14c6a799 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:26.324896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a8dd4-5cf1-467e-a0ed-081b14c6a799 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-7e0a8dd4-5cf1-467e-a0ed-081b14c6a799 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:26.341983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:26.344267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:26.346131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 407/818] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:26 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:26.348400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a8dd4-5cf1-467e-a0ed-081b14c6a799 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:26.433026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062075ca-ccb2-4e9a-9bb6-40e6eb9953da None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:26.473653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dafc35-0b30-4189-9c6e-666be7a49eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:26.474484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dafc35-0b30-4189-9c6e-666be7a49eda None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:26.474794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dafc35-0b30-4189-9c6e-666be7a49eda None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:26.475189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dafc35-0b30-4189-9c6e-666be7a49eda None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/5284fc12641540c3a42f291138b7c069/roles/d60298c676854e83b88df984e42773da` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:26.477300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a8dd4-5cf1-467e-a0ed-081b14c6a799 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:26.501267 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dafc35-0b30-4189-9c6e-666be7a49eda None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=5284fc12641540c3a42f291138b7c069, role_id=d60298c676854e83b88df984e42773da)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:26.502883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dafc35-0b30-4189-9c6e-666be7a49eda None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:26.519792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 412/819] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:12:26 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/5284fc12641540c3a42f291138b7c069/roles/d60298c676854e83b88df984e42773da => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:12:26.520864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dafc35-0b30-4189-9c6e-666be7a49eda None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:26.533603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c014a-276f-41fa-ba74-8e2f770d0bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:26.535083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c014a-276f-41fa-ba74-8e2f770d0bfa None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:26.535620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c014a-276f-41fa-ba74-8e2f770d0bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:26.536202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c014a-276f-41fa-ba74-8e2f770d0bfa None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:26.537805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c014a-276f-41fa-ba74-8e2f770d0bfa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:26.538490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c014a-276f-41fa-ba74-8e2f770d0bfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:26.542101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c014a-276f-41fa-ba74-8e2f770d0bfa None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:26.552114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 408/820] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Jul 20 08:12:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:26.553688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c014a-276f-41fa-ba74-8e2f770d0bfa None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:26.587392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:26.607325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:26.698964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dafc35-0b30-4189-9c6e-666be7a49eda None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:26.705095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c014a-276f-41fa-ba74-8e2f770d0bfa None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:26.752130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805b07c-6a73-4cc7-91b0-072a37726cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:26.753023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805b07c-6a73-4cc7-91b0-072a37726cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:26.754230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805b07c-6a73-4cc7-91b0-072a37726cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:26.754230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805b07c-6a73-4cc7-91b0-072a37726cb5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:26.759823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15662f3-3846-474d-92cf-feec1dbff8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:26.760533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15662f3-3846-474d-92cf-feec1dbff8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:26.760821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15662f3-3846-474d-92cf-feec1dbff8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:26.761103 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15662f3-3846-474d-92cf-feec1dbff8f3 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:26.761833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15662f3-3846-474d-92cf-feec1dbff8f3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:26.762144 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15662f3-3846-474d-92cf-feec1dbff8f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:26.763617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805b07c-6a73-4cc7-91b0-072a37726cb5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:26.763740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15662f3-3846-474d-92cf-feec1dbff8f3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:26.764932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805b07c-6a73-4cc7-91b0-072a37726cb5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:26.769943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 409/821] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:26.771477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15662f3-3846-474d-92cf-feec1dbff8f3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a805b07c-6a73-4cc7-91b0-072a37726cb5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:26.788685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:26.789855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:26.790974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:26.790974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:26.790974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:26.790974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:26.790974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:26.790974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:26.790974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:26.790974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:26.790974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:12:26.790974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:26.790974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 413/822] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Jul 20 08:12:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:26.791712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805b07c-6a73-4cc7-91b0-072a37726cb5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:26.889655 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:26.897906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:26.902525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15662f3-3846-474d-92cf-feec1dbff8f3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:26.912765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805b07c-6a73-4cc7-91b0-072a37726cb5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:26.943593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b19d20-164e-4c50-b2d4-41909b458314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:26.944998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b19d20-164e-4c50-b2d4-41909b458314 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:26.945441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b19d20-164e-4c50-b2d4-41909b458314 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:26.945922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b19d20-164e-4c50-b2d4-41909b458314 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:26.946907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b19d20-164e-4c50-b2d4-41909b458314 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:26.948646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b19d20-164e-4c50-b2d4-41909b458314 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:26.953620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d2785-c300-4e36-a693-42d1803f63ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:26.954829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d2785-c300-4e36-a693-42d1803f63ba None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:26.955233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d2785-c300-4e36-a693-42d1803f63ba None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:26.955518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d2785-c300-4e36-a693-42d1803f63ba None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:26.956278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d2785-c300-4e36-a693-42d1803f63ba None admin] RBAC: Adding query filter params (type=network) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:26.956563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d2785-c300-4e36-a693-42d1803f63ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:26.957952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d2785-c300-4e36-a693-42d1803f63ba None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:26.970801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 414/823] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:26.972278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d2785-c300-4e36-a693-42d1803f63ba None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:26.977585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 410/824] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Jul 20 08:12:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:26.978837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b19d20-164e-4c50-b2d4-41909b458314 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:27.101414 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d2785-c300-4e36-a693-42d1803f63ba None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:27.162835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27d474-f822-48c0-89f0-864e87f4e5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:27.163526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27d474-f822-48c0-89f0-864e87f4e5ae None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:27.163809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27d474-f822-48c0-89f0-864e87f4e5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:27.164086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27d474-f822-48c0-89f0-864e87f4e5ae None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:27.164956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27d474-f822-48c0-89f0-864e87f4e5ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:27.165266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27d474-f822-48c0-89f0-864e87f4e5ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:27.171237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e27d474-f822-48c0-89f0-864e87f4e5ae None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:27.214538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 415/825] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Jul 20 08:12:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:27.222450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27d474-f822-48c0-89f0-864e87f4e5ae None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:27.233977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:27.243202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b19d20-164e-4c50-b2d4-41909b458314 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:27.282495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd52292-e739-46ee-8891-d0b9764c4e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:27.283520 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd52292-e739-46ee-8891-d0b9764c4e9f None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:27.283958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd52292-e739-46ee-8891-d0b9764c4e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:27.284380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd52292-e739-46ee-8891-d0b9764c4e9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:27.285279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd52292-e739-46ee-8891-d0b9764c4e9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:27.286941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd52292-e739-46ee-8891-d0b9764c4e9f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:27.310515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 411/826] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Jul 20 08:12:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:27.367553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27d474-f822-48c0-89f0-864e87f4e5ae None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:27.439094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599105e-773e-4a13-8aa4-a2eefaa43e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:27.440132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599105e-773e-4a13-8aa4-a2eefaa43e6f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:27.440576 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599105e-773e-4a13-8aa4-a2eefaa43e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:27.441148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599105e-773e-4a13-8aa4-a2eefaa43e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:27.546339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599105e-773e-4a13-8aa4-a2eefaa43e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:27.548452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599105e-773e-4a13-8aa4-a2eefaa43e6f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:27.550943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:27.684752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 416/827] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:27 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 08:12:27.691030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd52292-e739-46ee-8891-d0b9764c4e9f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:27.698547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:27.825270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd52292-e739-46ee-8891-d0b9764c4e9f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:27.868057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830cc5a-a2e0-4303-b7e8-ee33a6079688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:27.868863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830cc5a-a2e0-4303-b7e8-ee33a6079688 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:27.869148 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830cc5a-a2e0-4303-b7e8-ee33a6079688 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:27.869420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830cc5a-a2e0-4303-b7e8-ee33a6079688 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:27.887655 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830cc5a-a2e0-4303-b7e8-ee33a6079688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:27.890518 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830cc5a-a2e0-4303-b7e8-ee33a6079688 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e830cc5a-a2e0-4303-b7e8-ee33a6079688 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:27.914995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:27.916487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:27.917353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:27.917353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:27.917353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:27.917353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:27.917353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 412/828] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:27.926628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599105e-773e-4a13-8aa4-a2eefaa43e6f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:28.152953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599105e-773e-4a13-8aa4-a2eefaa43e6f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:28.154542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:28.191309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a615a-0efb-4a0c-a35e-08a660eb78a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:28.191976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a615a-0efb-4a0c-a35e-08a660eb78a7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:28.192257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a615a-0efb-4a0c-a35e-08a660eb78a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:28.192529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21a615a-0efb-4a0c-a35e-08a660eb78a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:28.193251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a615a-0efb-4a0c-a35e-08a660eb78a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:28.193556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a615a-0efb-4a0c-a35e-08a660eb78a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:28.194962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21a615a-0efb-4a0c-a35e-08a660eb78a7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:28.207949 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 417/829] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:28.213862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830cc5a-a2e0-4303-b7e8-ee33a6079688 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:28.281081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:28.329345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830cc5a-a2e0-4303-b7e8-ee33a6079688 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:28.366800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a286e11-3f47-4a6c-98d2-a5db121f7c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:28.367970 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a286e11-3f47-4a6c-98d2-a5db121f7c55 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:28.368503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a286e11-3f47-4a6c-98d2-a5db121f7c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:28.369015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a286e11-3f47-4a6c-98d2-a5db121f7c55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:28.376731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a286e11-3f47-4a6c-98d2-a5db121f7c55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:28.378472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a286e11-3f47-4a6c-98d2-a5db121f7c55 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-0a286e11-3f47-4a6c-98d2-a5db121f7c55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:28.394616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:28.395612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:28.396469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:28.396469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:28.396469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:28.396469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:28.396469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:28.396469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:28.396469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:28.396469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:28.399080 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 413/830] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:28.406287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a615a-0efb-4a0c-a35e-08a660eb78a7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:28.509180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe993-5090-4968-b06e-823f89d962d8 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:28.509411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe993-5090-4968-b06e-823f89d962d8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:28.509690 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfe993-5090-4968-b06e-823f89d962d8 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:28.511391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 414/831] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:12:28.517049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206942b-bde7-47e1-a077-f04e41552eff None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:28.517488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206942b-bde7-47e1-a077-f04e41552eff None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:28.517770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206942b-bde7-47e1-a077-f04e41552eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:28.528089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21a615a-0efb-4a0c-a35e-08a660eb78a7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:28.537038 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1206942b-bde7-47e1-a077-f04e41552eff None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:28.538359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1206942b-bde7-47e1-a077-f04e41552eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:28.584105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331ea35-b58f-4227-9bc3-47a565a7f913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:28.585556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331ea35-b58f-4227-9bc3-47a565a7f913 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:28.586043 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331ea35-b58f-4227-9bc3-47a565a7f913 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:28.587071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331ea35-b58f-4227-9bc3-47a565a7f913 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:28.588096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331ea35-b58f-4227-9bc3-47a565a7f913 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:28.588634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331ea35-b58f-4227-9bc3-47a565a7f913 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:28.590354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7331ea35-b58f-4227-9bc3-47a565a7f913 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:28.613825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 418/832] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:28.616205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331ea35-b58f-4227-9bc3-47a565a7f913 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:28.634775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 415/833] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:28.645326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:28.655440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206942b-bde7-47e1-a077-f04e41552eff None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:28.714513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:28.753201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331ea35-b58f-4227-9bc3-47a565a7f913 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:28.785758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206942b-bde7-47e1-a077-f04e41552eff None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:28.792917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df322c6-9a4f-4fee-9d24-130e63eadb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:28.793694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df322c6-9a4f-4fee-9d24-130e63eadb06 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:28.793978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df322c6-9a4f-4fee-9d24-130e63eadb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:28.794253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df322c6-9a4f-4fee-9d24-130e63eadb06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:28.795127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df322c6-9a4f-4fee-9d24-130e63eadb06 None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=84f0da8709624834976d0662aeb02ea6) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:28.795436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df322c6-9a4f-4fee-9d24-130e63eadb06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:28.796890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df322c6-9a4f-4fee-9d24-130e63eadb06 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:28.810486 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 419/834] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:28 2022] GET /identity/v3/role_assignments?user.id=84f0da8709624834976d0662aeb02ea6&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:28.832046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca1a749-7c8b-4888-9143-bbf37f74c91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:28.833297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1a749-7c8b-4888-9143-bbf37f74c91e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:28.833815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1a749-7c8b-4888-9143-bbf37f74c91e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:28.834298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca1a749-7c8b-4888-9143-bbf37f74c91e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:28.848620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca1a749-7c8b-4888-9143-bbf37f74c91e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9ca1a749-7c8b-4888-9143-bbf37f74c91e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:28.857055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:28.858040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:28.858904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:28.863118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 416/835] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:12:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:28.869199 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df322c6-9a4f-4fee-9d24-130e63eadb06 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:28.878286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1eda24-27e8-4c35-bbcb-8ac2e35f9105 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:28.878650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1eda24-27e8-4c35-bbcb-8ac2e35f9105 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:28.878938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1eda24-27e8-4c35-bbcb-8ac2e35f9105 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:28.880788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 417/836] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:12:28.886039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d8b00-699a-45d9-ba30-7f7d6c18d1d2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:28.886370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d8b00-699a-45d9-ba30-7f7d6c18d1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:28.886659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d8b00-699a-45d9-ba30-7f7d6c18d1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:28.905089 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b05d8b00-699a-45d9-ba30-7f7d6c18d1d2 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:28.906350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b05d8b00-699a-45d9-ba30-7f7d6c18d1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:28.989967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:28.998642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 418/837] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:29.011173 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df322c6-9a4f-4fee-9d24-130e63eadb06 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:29.020941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d8b00-699a-45d9-ba30-7f7d6c18d1d2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:29.054056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa7dd1-d3ec-48d0-af2e-d988bf51677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:29.054785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa7dd1-d3ec-48d0-af2e-d988bf51677b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:29.055076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa7dd1-d3ec-48d0-af2e-d988bf51677b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:29.055368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa7dd1-d3ec-48d0-af2e-d988bf51677b None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:29.056112 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa7dd1-d3ec-48d0-af2e-d988bf51677b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:29.056504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa7dd1-d3ec-48d0-af2e-d988bf51677b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:29.057807 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa7dd1-d3ec-48d0-af2e-d988bf51677b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:29.062807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:29.068858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 420/838] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:12:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:29.075406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa7dd1-d3ec-48d0-af2e-d988bf51677b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:29.164836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d8b00-699a-45d9-ba30-7f7d6c18d1d2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:29.205104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3bf1b-c686-4342-b871-1afcf098ce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:29.205985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3bf1b-c686-4342-b871-1afcf098ce63 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:29.206273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3bf1b-c686-4342-b871-1afcf098ce63 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:29.206613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3bf1b-c686-4342-b871-1afcf098ce63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:29.210651 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa7dd1-d3ec-48d0-af2e-d988bf51677b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:29.220153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e3bf1b-c686-4342-b871-1afcf098ce63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-83e3bf1b-c686-4342-b871-1afcf098ce63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:29.228321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:29.229242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:12:29.230046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:29.234624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 419/839] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:12:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:29.237715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3bf1b-c686-4342-b871-1afcf098ce63 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:29.264151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa0a8b-565f-4368-9e3a-1ce781befa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:29.265662 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa0a8b-565f-4368-9e3a-1ce781befa30 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:29.266156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa0a8b-565f-4368-9e3a-1ce781befa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:29.266855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa0a8b-565f-4368-9e3a-1ce781befa30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:29.283732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa0a8b-565f-4368-9e3a-1ce781befa30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:29.285465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa0a8b-565f-4368-9e3a-1ce781befa30 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:29.298708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-36fa0a8b-565f-4368-9e3a-1ce781befa30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:29.314884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:29.316026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:29.317121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:29.317121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:29.317121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:29.317121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:29.318614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 421/840] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:29.323747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa0a8b-565f-4368-9e3a-1ce781befa30 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:29.373786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:29.384103 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3bf1b-c686-4342-b871-1afcf098ce63 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:29.434863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269d47-001a-4576-bdb4-8b247c05c2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:29.436123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269d47-001a-4576-bdb4-8b247c05c2ac None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:29.436725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269d47-001a-4576-bdb4-8b247c05c2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:29.437326 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af269d47-001a-4576-bdb4-8b247c05c2ac None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:29.439055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af269d47-001a-4576-bdb4-8b247c05c2ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:29.439856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af269d47-001a-4576-bdb4-8b247c05c2ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:29.442204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af269d47-001a-4576-bdb4-8b247c05c2ac None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:29.443524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa0a8b-565f-4368-9e3a-1ce781befa30 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:29.455879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 420/841] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:12:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:29.461359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269d47-001a-4576-bdb4-8b247c05c2ac None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:29.494422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7561f23-ee58-4776-b5b3-e6fce67cf1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:29.495439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7561f23-ee58-4776-b5b3-e6fce67cf1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:29.496099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7561f23-ee58-4776-b5b3-e6fce67cf1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:29.496484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7561f23-ee58-4776-b5b3-e6fce67cf1c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:29.497784 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7561f23-ee58-4776-b5b3-e6fce67cf1c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:29.498338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7561f23-ee58-4776-b5b3-e6fce67cf1c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:29.500251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7561f23-ee58-4776-b5b3-e6fce67cf1c8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:29.524572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 422/842] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:29.526440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7561f23-ee58-4776-b5b3-e6fce67cf1c8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:29.602567 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af269d47-001a-4576-bdb4-8b247c05c2ac None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:29.603186 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:29.644272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb0ab8-e469-4843-b434-dbd412a4e1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:29.645073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb0ab8-e469-4843-b434-dbd412a4e1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:29.645360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb0ab8-e469-4843-b434-dbd412a4e1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:29.645635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb0ab8-e469-4843-b434-dbd412a4e1f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:29.653393 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7561f23-ee58-4776-b5b3-e6fce67cf1c8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:29.659107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb0ab8-e469-4843-b434-dbd412a4e1f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:29.660681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb0ab8-e469-4843-b434-dbd412a4e1f0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-07fb0ab8-e469-4843-b434-dbd412a4e1f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:29.680941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:29.682098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:29.683190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:29.683190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:29.683190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:29.683190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:29.683190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 421/843] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:29.683887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b212a-0a92-4152-97ef-ded94572ee58 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:29.684185 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b212a-0a92-4152-97ef-ded94572ee58 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:29.684467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b212a-0a92-4152-97ef-ded94572ee58 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:29.685945 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 422/844] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:29.692036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b212a-0a92-4152-97ef-ded94572ee58 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:29.701379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead14238-842a-4451-adcb-6d81a6d27632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:29.702192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14238-842a-4451-adcb-6d81a6d27632 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:29.702477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14238-842a-4451-adcb-6d81a6d27632 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:29.702790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead14238-842a-4451-adcb-6d81a6d27632 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:29.710776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:29.712553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead14238-842a-4451-adcb-6d81a6d27632 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:29.714453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead14238-842a-4451-adcb-6d81a6d27632 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ead14238-842a-4451-adcb-6d81a6d27632 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:29.736423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:29.737485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:29.738423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:29.738423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:29.738423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:29.738423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:29.738423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:29.738423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:29.738423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:29.738423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:29.738423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 08:12:29.738423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:29.738423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 423/845] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:12:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:29.739699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4670ce-1d00-4019-a4be-cfbfcdc9dfa4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:29.739699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4670ce-1d00-4019-a4be-cfbfcdc9dfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:29.739875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4670ce-1d00-4019-a4be-cfbfcdc9dfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:29.761308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bc4670ce-1d00-4019-a4be-cfbfcdc9dfa4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:29.762534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bc4670ce-1d00-4019-a4be-cfbfcdc9dfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:29.829262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b212a-0a92-4152-97ef-ded94572ee58 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:29.858607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 424/846] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:29.858607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4670ce-1d00-4019-a4be-cfbfcdc9dfa4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:29.886147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd2e21-9ed0-4d31-8592-ef308dacbea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:29.887029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd2e21-9ed0-4d31-8592-ef308dacbea0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:29.887320 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd2e21-9ed0-4d31-8592-ef308dacbea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:29.887603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd2e21-9ed0-4d31-8592-ef308dacbea0 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:29.888339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd2e21-9ed0-4d31-8592-ef308dacbea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:29.888650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd2e21-9ed0-4d31-8592-ef308dacbea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:29.890086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd2e21-9ed0-4d31-8592-ef308dacbea0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:29.907059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 423/847] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:29.907059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd2e21-9ed0-4d31-8592-ef308dacbea0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:29.923341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:29.987313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4670ce-1d00-4019-a4be-cfbfcdc9dfa4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:29.989510 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:30.029317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73618214-d636-4efd-aa9c-a8c931e93da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:30.029994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73618214-d636-4efd-aa9c-a8c931e93da8 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:30.030286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73618214-d636-4efd-aa9c-a8c931e93da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:30.030566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73618214-d636-4efd-aa9c-a8c931e93da8 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:30.031449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73618214-d636-4efd-aa9c-a8c931e93da8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:30.031848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73618214-d636-4efd-aa9c-a8c931e93da8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:30.034132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73618214-d636-4efd-aa9c-a8c931e93da8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:30.042514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd2e21-9ed0-4d31-8592-ef308dacbea0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:30.055221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 425/848] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Jul 20 08:12:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:30.056077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73618214-d636-4efd-aa9c-a8c931e93da8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:30.091709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb5fbf-88ac-48de-8cf1-84e5925ce648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:30.092419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5fbf-88ac-48de-8cf1-84e5925ce648 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:30.092706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5fbf-88ac-48de-8cf1-84e5925ce648 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:30.092987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5fbf-88ac-48de-8cf1-84e5925ce648 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:30.093669 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb5fbf-88ac-48de-8cf1-84e5925ce648 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:30.095320 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb5fbf-88ac-48de-8cf1-84e5925ce648 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-5ffb5fbf-88ac-48de-8cf1-84e5925ce648 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:30.107950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:30.109314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:30.110338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 424/849] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:12:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:30.112307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb5fbf-88ac-48de-8cf1-84e5925ce648 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:30.186597 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73618214-d636-4efd-aa9c-a8c931e93da8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:30.198505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:30.229398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f772da-c752-4909-8e75-1627bf25629d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:30.229726 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb5fbf-88ac-48de-8cf1-84e5925ce648 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:30.230413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f772da-c752-4909-8e75-1627bf25629d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:30.230556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f772da-c752-4909-8e75-1627bf25629d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:30.230928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f772da-c752-4909-8e75-1627bf25629d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:30.240820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f772da-c752-4909-8e75-1627bf25629d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:30.242657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f772da-c752-4909-8e75-1627bf25629d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-f5f772da-c752-4909-8e75-1627bf25629d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:30.259231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:30.260287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:30.261296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:30.261296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:30.261296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:30.261296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:30.261296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:30.261296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:30.261296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:30.261296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:30.261296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:30.261296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:30.265034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 426/850] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Jul 20 08:12:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:30.265959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f772da-c752-4909-8e75-1627bf25629d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:30.273235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4171f-bdf9-4e6f-8ef1-c926b7de65aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:30.273939 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4171f-bdf9-4e6f-8ef1-c926b7de65aa None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:30.274180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4171f-bdf9-4e6f-8ef1-c926b7de65aa None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:30.274453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4171f-bdf9-4e6f-8ef1-c926b7de65aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:30.278172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:30.356880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4171f-bdf9-4e6f-8ef1-c926b7de65aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:30.358661 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4171f-bdf9-4e6f-8ef1-c926b7de65aa None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:30.387991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f772da-c752-4909-8e75-1627bf25629d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:30.430919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951cab0-a6ed-4bc1-a482-2a7712f48dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:30.431576 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951cab0-a6ed-4bc1-a482-2a7712f48dea None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:30.431859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951cab0-a6ed-4bc1-a482-2a7712f48dea None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:30.432193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951cab0-a6ed-4bc1-a482-2a7712f48dea None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:30.432856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951cab0-a6ed-4bc1-a482-2a7712f48dea None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:30.433158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951cab0-a6ed-4bc1-a482-2a7712f48dea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:30.434560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951cab0-a6ed-4bc1-a482-2a7712f48dea None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:30.441890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 427/851] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:12:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:30.442803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951cab0-a6ed-4bc1-a482-2a7712f48dea None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:30.483061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:30.494908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 425/852] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:12:30.495793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4171f-bdf9-4e6f-8ef1-c926b7de65aa None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:30.541026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:30.575622 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951cab0-a6ed-4bc1-a482-2a7712f48dea None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:30.626052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ca3bc-ac53-49d5-9a3d-b462ada5ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:30.626698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ca3bc-ac53-49d5-9a3d-b462ada5ce3a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:30.627289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ca3bc-ac53-49d5-9a3d-b462ada5ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:30.627289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3ca3bc-ac53-49d5-9a3d-b462ada5ce3a None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:30.628090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ca3bc-ac53-49d5-9a3d-b462ada5ce3a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:30.628406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ca3bc-ac53-49d5-9a3d-b462ada5ce3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:30.629874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3ca3bc-ac53-49d5-9a3d-b462ada5ce3a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:30.630864 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4171f-bdf9-4e6f-8ef1-c926b7de65aa None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:30.655627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 428/853] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Jul 20 08:12:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:30.655627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ca3bc-ac53-49d5-9a3d-b462ada5ce3a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:30.684993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e039bf7-400d-4453-bda8-678136d1b7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:30.685791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e039bf7-400d-4453-bda8-678136d1b7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:30.686143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e039bf7-400d-4453-bda8-678136d1b7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:30.686423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e039bf7-400d-4453-bda8-678136d1b7f9 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:30.687173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e039bf7-400d-4453-bda8-678136d1b7f9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:30.687484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e039bf7-400d-4453-bda8-678136d1b7f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:30.689041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e039bf7-400d-4453-bda8-678136d1b7f9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:30.702620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 426/854] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:12:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:30.702620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e039bf7-400d-4453-bda8-678136d1b7f9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:30.784040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:30.797605 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3ca3bc-ac53-49d5-9a3d-b462ada5ce3a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:30.848489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa073dd-9e59-4e74-abf8-f5f7ca9c5624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:30.849313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa073dd-9e59-4e74-abf8-f5f7ca9c5624 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:30.849729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa073dd-9e59-4e74-abf8-f5f7ca9c5624 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:30.850017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa073dd-9e59-4e74-abf8-f5f7ca9c5624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:30.864836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa073dd-9e59-4e74-abf8-f5f7ca9c5624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:30.866499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa073dd-9e59-4e74-abf8-f5f7ca9c5624 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:30.871940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:30.878395 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e039bf7-400d-4453-bda8-678136d1b7f9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-2aa073dd-9e59-4e74-abf8-f5f7ca9c5624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:30.896396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:30.897371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:30.898231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:30.898231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:30.898231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:30.898231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:30.900820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 429/855] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:30.908509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa073dd-9e59-4e74-abf8-f5f7ca9c5624 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:30.929963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab94014-2ec7-4f11-bba9-d822d4d87c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:30.931119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab94014-2ec7-4f11-bba9-d822d4d87c6d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:30.932459 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab94014-2ec7-4f11-bba9-d822d4d87c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:30.932459 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab94014-2ec7-4f11-bba9-d822d4d87c6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:31.046229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab94014-2ec7-4f11-bba9-d822d4d87c6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:31.046616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab94014-2ec7-4f11-bba9-d822d4d87c6d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:31.046719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa073dd-9e59-4e74-abf8-f5f7ca9c5624 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:31.084753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfa7da-29ad-4ecb-b78a-43d7a9b27b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:31.085425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfa7da-29ad-4ecb-b78a-43d7a9b27b09 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:31.085712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfa7da-29ad-4ecb-b78a-43d7a9b27b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:31.085988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfa7da-29ad-4ecb-b78a-43d7a9b27b09 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:31.086749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfa7da-29ad-4ecb-b78a-43d7a9b27b09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:31.087060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfa7da-29ad-4ecb-b78a-43d7a9b27b09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:31.088459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfa7da-29ad-4ecb-b78a-43d7a9b27b09 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:31.102270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 430/856] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:31.104143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c2313-ecb2-4753-8d66-0847e0e9a8bd None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:31.104613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c2313-ecb2-4753-8d66-0847e0e9a8bd None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:31.105050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c2313-ecb2-4753-8d66-0847e0e9a8bd None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:31.106433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 431/857] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:31.112234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c2313-ecb2-4753-8d66-0847e0e9a8bd None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:31.115197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:31.164590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:31.169314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 427/858] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:12:31.170992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b1958-c540-48ac-8f4e-ac14ed659003 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:31.171707 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b1958-c540-48ac-8f4e-ac14ed659003 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:31.171987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b1958-c540-48ac-8f4e-ac14ed659003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:31.190396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-063b1958-c540-48ac-8f4e-ac14ed659003 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:31.191636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-063b1958-c540-48ac-8f4e-ac14ed659003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:31.232285 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c2313-ecb2-4753-8d66-0847e0e9a8bd None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:31.273757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 428/859] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:31.274302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaad81-5e58-4941-9d23-7663d0dc128b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:31.274545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaad81-5e58-4941-9d23-7663d0dc128b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:31.274793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaad81-5e58-4941-9d23-7663d0dc128b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:31.275067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaad81-5e58-4941-9d23-7663d0dc128b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:31.279533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b1958-c540-48ac-8f4e-ac14ed659003 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:31.283342 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aaad81-5e58-4941-9d23-7663d0dc128b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:31.285002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aaad81-5e58-4941-9d23-7663d0dc128b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d2aaad81-5e58-4941-9d23-7663d0dc128b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:31.304015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:31.304958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:31.305811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:31.305811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:31.305811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:31.305811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:31.305811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:31.305811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:31.305811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:31.305811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:31.308011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 432/860] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:31.309007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaad81-5e58-4941-9d23-7663d0dc128b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:31.382776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:31.409143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b1958-c540-48ac-8f4e-ac14ed659003 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:31.430004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaad81-5e58-4941-9d23-7663d0dc128b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:31.437308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:31.447712 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3c5bb-6989-4a2c-8c11-e95f353b7ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:31.448683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3c5bb-6989-4a2c-8c11-e95f353b7ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:31.449067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3c5bb-6989-4a2c-8c11-e95f353b7ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:31.449440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b3c5bb-6989-4a2c-8c11-e95f353b7ac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:31.461263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3c5bb-6989-4a2c-8c11-e95f353b7ac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:31.463127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b3c5bb-6989-4a2c-8c11-e95f353b7ac0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:31.472938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21b72a-5cfb-4360-af22-6aed0198ca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:31.474767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21b72a-5cfb-4360-af22-6aed0198ca8e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:31.475145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21b72a-5cfb-4360-af22-6aed0198ca8e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:31.475514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21b72a-5cfb-4360-af22-6aed0198ca8e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:31.476271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21b72a-5cfb-4360-af22-6aed0198ca8e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:31.477749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d21b72a-5cfb-4360-af22-6aed0198ca8e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d6b3c5bb-6989-4a2c-8c11-e95f353b7ac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:31.482905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:31.484076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:31.486115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:31.486115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:31.486115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:31.486115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:31.486115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 429/861] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:31.486115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3c5bb-6989-4a2c-8c11-e95f353b7ac0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0d21b72a-5cfb-4360-af22-6aed0198ca8e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:31.497428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:31.498989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 08:12:31.500030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:31.503166 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 433/862] 10.222.0.64 () {56 vars in 1132 bytes} [Wed Jul 20 08:12:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:31.503166 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21b72a-5cfb-4360-af22-6aed0198ca8e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:31.608594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b3c5bb-6989-4a2c-8c11-e95f353b7ac0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:31.618206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21b72a-5cfb-4360-af22-6aed0198ca8e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:31.644978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293a5d1-2b6a-4341-a1e6-6363ea4826ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:31.645645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293a5d1-2b6a-4341-a1e6-6363ea4826ef None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:31.645936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293a5d1-2b6a-4341-a1e6-6363ea4826ef None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:31.646215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293a5d1-2b6a-4341-a1e6-6363ea4826ef None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:31.646807 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:31.647252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293a5d1-2b6a-4341-a1e6-6363ea4826ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:31.647339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293a5d1-2b6a-4341-a1e6-6363ea4826ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:31.648824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293a5d1-2b6a-4341-a1e6-6363ea4826ef None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:31.663631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca870ad-f836-4f7f-9031-4f3f62be57c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:31.664003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 430/863] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:31.664831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293a5d1-2b6a-4341-a1e6-6363ea4826ef None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:31.669775 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca870ad-f836-4f7f-9031-4f3f62be57c9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:31.670111 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca870ad-f836-4f7f-9031-4f3f62be57c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:31.670390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca870ad-f836-4f7f-9031-4f3f62be57c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:31.671159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca870ad-f836-4f7f-9031-4f3f62be57c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:31.671469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca870ad-f836-4f7f-9031-4f3f62be57c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:31.673538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca870ad-f836-4f7f-9031-4f3f62be57c9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:31.687678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 434/864] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:31.689060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca870ad-f836-4f7f-9031-4f3f62be57c9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:31.714565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:31.780359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293a5d1-2b6a-4341-a1e6-6363ea4826ef None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:31.807029 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca870ad-f836-4f7f-9031-4f3f62be57c9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:31.821792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98945371-50b0-4676-b6a7-b97f8403449e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:31.822474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98945371-50b0-4676-b6a7-b97f8403449e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:31.822791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98945371-50b0-4676-b6a7-b97f8403449e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:31.823245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98945371-50b0-4676-b6a7-b97f8403449e None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:31.823921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98945371-50b0-4676-b6a7-b97f8403449e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:31.824319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98945371-50b0-4676-b6a7-b97f8403449e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:31.825707 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98945371-50b0-4676-b6a7-b97f8403449e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:31.836223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 431/865] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Jul 20 08:12:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:31.836223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98945371-50b0-4676-b6a7-b97f8403449e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:31.852759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4d5e3-69cd-43b3-8ac4-ae995be15f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:31.853421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4d5e3-69cd-43b3-8ac4-ae995be15f30 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:31.853738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4d5e3-69cd-43b3-8ac4-ae995be15f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:31.854015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4d5e3-69cd-43b3-8ac4-ae995be15f30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:31.854999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4d5e3-69cd-43b3-8ac4-ae995be15f30 None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=c3cf87f0515b460dad8c0ff5591c2685) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:31.855494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4d5e3-69cd-43b3-8ac4-ae995be15f30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:31.856873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a4d5e3-69cd-43b3-8ac4-ae995be15f30 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:31.874912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 435/866] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:31 2022] GET /identity/v3/role_assignments?user.id=c3cf87f0515b460dad8c0ff5591c2685&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:31.874912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4d5e3-69cd-43b3-8ac4-ae995be15f30 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:31.949344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:31.980208 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98945371-50b0-4676-b6a7-b97f8403449e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:31.995266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:32.036592 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4d5e3-69cd-43b3-8ac4-ae995be15f30 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:32.037356 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7c60e-6413-4718-89e1-93339e5252a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:32.038886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7c60e-6413-4718-89e1-93339e5252a5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:32.039343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7c60e-6413-4718-89e1-93339e5252a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:32.040819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7c60e-6413-4718-89e1-93339e5252a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:32.050052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7c60e-6413-4718-89e1-93339e5252a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:32.052233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7c60e-6413-4718-89e1-93339e5252a5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-40e7c60e-6413-4718-89e1-93339e5252a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:32.070916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:32.071861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:32.072719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:32.072719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:32.072719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:32.072719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:32.072719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:32.072719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:32.072719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:32.072719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:32.072719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 432/867] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:32.078291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7c60e-6413-4718-89e1-93339e5252a5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:32.087601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25956b-5f72-49a1-a7d3-6e1f3aff2b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:32.088631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25956b-5f72-49a1-a7d3-6e1f3aff2b26 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:32.089114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25956b-5f72-49a1-a7d3-6e1f3aff2b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:32.090047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25956b-5f72-49a1-a7d3-6e1f3aff2b26 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:32.091092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc25956b-5f72-49a1-a7d3-6e1f3aff2b26 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:32.091305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc25956b-5f72-49a1-a7d3-6e1f3aff2b26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:32.093363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc25956b-5f72-49a1-a7d3-6e1f3aff2b26 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:32.100975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 436/868] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Jul 20 08:12:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:32.201464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7c60e-6413-4718-89e1-93339e5252a5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:32.244048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62974dd-e3b9-41cd-88e6-1c70dafc7f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:32.244743 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62974dd-e3b9-41cd-88e6-1c70dafc7f19 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:32.245035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62974dd-e3b9-41cd-88e6-1c70dafc7f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:32.245315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62974dd-e3b9-41cd-88e6-1c70dafc7f19 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:32.246189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62974dd-e3b9-41cd-88e6-1c70dafc7f19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:32.246507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62974dd-e3b9-41cd-88e6-1c70dafc7f19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:32.248039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62974dd-e3b9-41cd-88e6-1c70dafc7f19 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:32.264200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 433/869] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:32.271064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25956b-5f72-49a1-a7d3-6e1f3aff2b26 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:32.395607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25956b-5f72-49a1-a7d3-6e1f3aff2b26 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:32.411621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:32.441719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7c8fc-465d-44a3-8005-a93f60c64ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:32.442928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7c8fc-465d-44a3-8005-a93f60c64ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:32.443454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7c8fc-465d-44a3-8005-a93f60c64ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:32.443934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7c8fc-465d-44a3-8005-a93f60c64ab6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:32.444991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7c8fc-465d-44a3-8005-a93f60c64ab6 None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=5284fc12641540c3a42f291138b7c069) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:32.445511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7c8fc-465d-44a3-8005-a93f60c64ab6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:32.447259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7c8fc-465d-44a3-8005-a93f60c64ab6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:32.454805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 437/870] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:32 2022] GET /identity/v3/role_assignments?user.id=5284fc12641540c3a42f291138b7c069&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:32.984029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c09bab-5526-4195-b290-4192f06a6b09 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:32.984549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c09bab-5526-4195-b290-4192f06a6b09 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:32.984650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c09bab-5526-4195-b290-4192f06a6b09 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:32.987202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 434/871] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:32.993072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e602b0-410d-4db1-ae48-bac940a5d6d6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:32.993554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e602b0-410d-4db1-ae48-bac940a5d6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:32.993629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e602b0-410d-4db1-ae48-bac940a5d6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:33.011973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a2e602b0-410d-4db1-ae48-bac940a5d6d6 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:33.013300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a2e602b0-410d-4db1-ae48-bac940a5d6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:33.094825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 438/872] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:33.114136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c09bab-5526-4195-b290-4192f06a6b09 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:33.148928 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:33.244922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c09bab-5526-4195-b290-4192f06a6b09 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:33.288231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a747c15-a398-4609-9c33-c8a266acccc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:33.289135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a747c15-a398-4609-9c33-c8a266acccc9 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:33.289452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a747c15-a398-4609-9c33-c8a266acccc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:33.289693 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a747c15-a398-4609-9c33-c8a266acccc9 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:33.290757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a747c15-a398-4609-9c33-c8a266acccc9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:33.292147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a747c15-a398-4609-9c33-c8a266acccc9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:33.312047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 435/873] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:12:33 2022] POST /identity/v3/services => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:34.133623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decf65a-7e52-4dbe-9a4c-9a316e1ccb27 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:34.134160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decf65a-7e52-4dbe-9a4c-9a316e1ccb27 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:34.134160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decf65a-7e52-4dbe-9a4c-9a316e1ccb27 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:34.135859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 439/874] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:34.141794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38245038-5ebc-45af-92c9-4bf1eb4750dd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:34.142420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38245038-5ebc-45af-92c9-4bf1eb4750dd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:34.142420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38245038-5ebc-45af-92c9-4bf1eb4750dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:34.159927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-38245038-5ebc-45af-92c9-4bf1eb4750dd None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:34.161178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-38245038-5ebc-45af-92c9-4bf1eb4750dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:34.255194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 436/875] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:34.277862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decf65a-7e52-4dbe-9a4c-9a316e1ccb27 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:34.375340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5fe04-e976-44db-b837-f700a04aa79c None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:34.376597 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5fe04-e976-44db-b837-f700a04aa79c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:34.377104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5fe04-e976-44db-b837-f700a04aa79c None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:34.378996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 437/876] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:12:34.384106 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa448bf4-7ab0-4c8b-84be-e7371e751d22 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:34.384493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa448bf4-7ab0-4c8b-84be-e7371e751d22 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:34.384779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa448bf4-7ab0-4c8b-84be-e7371e751d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:34.395475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decf65a-7e52-4dbe-9a4c-9a316e1ccb27 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:34.403261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-fa448bf4-7ab0-4c8b-84be-e7371e751d22 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:34.404586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fa448bf4-7ab0-4c8b-84be-e7371e751d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:34.406364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:34.409887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:34.435620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ef42e-c73f-4520-a2b0-9ec5c6bc19dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:34.436448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef42e-c73f-4520-a2b0-9ec5c6bc19dd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:34.436730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef42e-c73f-4520-a2b0-9ec5c6bc19dd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:34.437004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef42e-c73f-4520-a2b0-9ec5c6bc19dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:34.437730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ef42e-c73f-4520-a2b0-9ec5c6bc19dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:34.439725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ef42e-c73f-4520-a2b0-9ec5c6bc19dd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-845ef42e-c73f-4520-a2b0-9ec5c6bc19dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:34.452343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:34.453332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:34.454623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 440/877] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Jul 20 08:12:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:34.459853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ef42e-c73f-4520-a2b0-9ec5c6bc19dd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:34.506605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 438/878] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:34 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:34.524975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa448bf4-7ab0-4c8b-84be-e7371e751d22 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:34.590811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ef42e-c73f-4520-a2b0-9ec5c6bc19dd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:34.636597 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea7dda-d614-4902-aab8-2e6ec492163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:34.637078 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea7dda-d614-4902-aab8-2e6ec492163b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:34.637364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea7dda-d614-4902-aab8-2e6ec492163b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:34.637656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deea7dda-d614-4902-aab8-2e6ec492163b None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:34.638364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea7dda-d614-4902-aab8-2e6ec492163b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:34.638713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea7dda-d614-4902-aab8-2e6ec492163b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:34.640665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deea7dda-d614-4902-aab8-2e6ec492163b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:34.648069 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 441/879] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Jul 20 08:12:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:34.653390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea7dda-d614-4902-aab8-2e6ec492163b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:34.678829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa448bf4-7ab0-4c8b-84be-e7371e751d22 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:34.707530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:34.730411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa383281-3c2c-4720-b7dd-bad50a1dc3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:34.732134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383281-3c2c-4720-b7dd-bad50a1dc3ff None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:34.732623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383281-3c2c-4720-b7dd-bad50a1dc3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:34.733362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383281-3c2c-4720-b7dd-bad50a1dc3ff None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:34.734543 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa383281-3c2c-4720-b7dd-bad50a1dc3ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:34.738975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa383281-3c2c-4720-b7dd-bad50a1dc3ff None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:34.750061 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:34.767169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 439/880] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:12:34 2022] POST /identity/v3/services => generated 269 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:34.768724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d9cbe-28f9-41e2-9173-0d82ba7ca16b None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:34.769011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d9cbe-28f9-41e2-9173-0d82ba7ca16b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:34.769283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d9cbe-28f9-41e2-9173-0d82ba7ca16b None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:34.770814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 440/881] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:34.776665 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acd872-7ce3-49b7-bda9-6f8e74fcd9e5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:34.777297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acd872-7ce3-49b7-bda9-6f8e74fcd9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:34.777781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acd872-7ce3-49b7-bda9-6f8e74fcd9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:34.797336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deea7dda-d614-4902-aab8-2e6ec492163b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:34.804264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-48acd872-7ce3-49b7-bda9-6f8e74fcd9e5 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:34.805902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-48acd872-7ce3-49b7-bda9-6f8e74fcd9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:34.856038 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a300c93-0c8a-4b96-906f-64ea101e257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:34.856870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a300c93-0c8a-4b96-906f-64ea101e257d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:34.857210 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a300c93-0c8a-4b96-906f-64ea101e257d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:34.857496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a300c93-0c8a-4b96-906f-64ea101e257d None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:34.858228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a300c93-0c8a-4b96-906f-64ea101e257d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:34.858546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a300c93-0c8a-4b96-906f-64ea101e257d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:34.862326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a300c93-0c8a-4b96-906f-64ea101e257d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:34.870750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 442/882] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Jul 20 08:12:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:34.910375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 441/883] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:34.945933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a300c93-0c8a-4b96-906f-64ea101e257d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:35.112956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:35.120681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a300c93-0c8a-4b96-906f-64ea101e257d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:35.161957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927a99b-9c67-4793-b9d1-0a743229e59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:35.162811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927a99b-9c67-4793-b9d1-0a743229e59b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:35.163093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927a99b-9c67-4793-b9d1-0a743229e59b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:35.163361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927a99b-9c67-4793-b9d1-0a743229e59b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:35.175013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5927a99b-9c67-4793-b9d1-0a743229e59b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5927a99b-9c67-4793-b9d1-0a743229e59b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:35.183272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:35.184185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:35.185002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 443/884] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Jul 20 08:12:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:35.191922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acd872-7ce3-49b7-bda9-6f8e74fcd9e5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:35.302206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acd872-7ce3-49b7-bda9-6f8e74fcd9e5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:35.335890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:35.342703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aacffe9-d545-4bc4-b6c2-ca2abf57ee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:35.343389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacffe9-d545-4bc4-b6c2-ca2abf57ee4e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:35.343674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacffe9-d545-4bc4-b6c2-ca2abf57ee4e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:35.344025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aacffe9-d545-4bc4-b6c2-ca2abf57ee4e None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:35.345413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacffe9-d545-4bc4-b6c2-ca2abf57ee4e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:35.345763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacffe9-d545-4bc4-b6c2-ca2abf57ee4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:35.347145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aacffe9-d545-4bc4-b6c2-ca2abf57ee4e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:35.356995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 442/885] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Jul 20 08:12:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:35.363784 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927a99b-9c67-4793-b9d1-0a743229e59b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:35.475574 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927a99b-9c67-4793-b9d1-0a743229e59b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:35.514007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed7c3d-d2ae-4a3d-9836-efb015b9f2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:35.514847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed7c3d-d2ae-4a3d-9836-efb015b9f2fb None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:35.515130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed7c3d-d2ae-4a3d-9836-efb015b9f2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:35.515405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed7c3d-d2ae-4a3d-9836-efb015b9f2fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:35.527175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed7c3d-d2ae-4a3d-9836-efb015b9f2fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:35.528655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed7c3d-d2ae-4a3d-9836-efb015b9f2fb None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:35.539830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18a06f-930a-4277-87ca-bfdd0fccc59c None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:35.540124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18a06f-930a-4277-87ca-bfdd0fccc59c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:35.540407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18a06f-930a-4277-87ca-bfdd0fccc59c None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:35.542081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 443/886] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:35.547911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a35c1-924c-42a3-8426-b6c74ef7c905 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c6ed7c3d-d2ae-4a3d-9836-efb015b9f2fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:35.548072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:35.548892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:35.549870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:35.549870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:35.549870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:35.549870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:35.550086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a35c1-924c-42a3-8426-b6c74ef7c905 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:35.550086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5a35c1-924c-42a3-8426-b6c74ef7c905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:35.550226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 444/887] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:35.555557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed7c3d-d2ae-4a3d-9836-efb015b9f2fb None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:35.571425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-dc5a35c1-924c-42a3-8426-b6c74ef7c905 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:35.572792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dc5a35c1-924c-42a3-8426-b6c74ef7c905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:35.575453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:35.654641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 444/888] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:35.682685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a35c1-924c-42a3-8426-b6c74ef7c905 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:35.704977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed7c3d-d2ae-4a3d-9836-efb015b9f2fb None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:35.742027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2556a-e8d3-4701-bccf-c3503e00a519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:35.742756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2556a-e8d3-4701-bccf-c3503e00a519 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:35.743039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2556a-e8d3-4701-bccf-c3503e00a519 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:35.743309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2556a-e8d3-4701-bccf-c3503e00a519 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:35.744049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2556a-e8d3-4701-bccf-c3503e00a519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:35.744352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2556a-e8d3-4701-bccf-c3503e00a519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:35.746031 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2556a-e8d3-4701-bccf-c3503e00a519 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:35.759551 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 445/889] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:35.764771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2556a-e8d3-4701-bccf-c3503e00a519 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:35.802087 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5a35c1-924c-42a3-8426-b6c74ef7c905 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:35.827076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:35.838304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:35.843649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbcd85-da98-470c-8e34-62354bb738d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:35.846311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbcd85-da98-470c-8e34-62354bb738d2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:35.847182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbcd85-da98-470c-8e34-62354bb738d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:35.847953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fbcd85-da98-470c-8e34-62354bb738d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:35.849495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbcd85-da98-470c-8e34-62354bb738d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:35.851922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fbcd85-da98-470c-8e34-62354bb738d2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-16fbcd85-da98-470c-8e34-62354bb738d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:35.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:35.868675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 08:12:35.869490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:35.878233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 445/890] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:12:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:35.879494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbcd85-da98-470c-8e34-62354bb738d2 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:35.879675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2556a-e8d3-4701-bccf-c3503e00a519 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:35.921102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bfe7b-2c1d-4e54-b623-9edd1b9e2370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:35.921317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfe7b-2c1d-4e54-b623-9edd1b9e2370 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:35.921585 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfe7b-2c1d-4e54-b623-9edd1b9e2370 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:35.921822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bfe7b-2c1d-4e54-b623-9edd1b9e2370 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:35.931352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bfe7b-2c1d-4e54-b623-9edd1b9e2370 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:35.932918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7bfe7b-2c1d-4e54-b623-9edd1b9e2370 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-eb7bfe7b-2c1d-4e54-b623-9edd1b9e2370 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:35.950559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:35.951561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:35.952515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:35.952515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:35.952515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:35.952515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:35.952515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:35.952515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:35.952515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:35.952515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:35.952515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:35.952515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:35.957489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 446/891] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Jul 20 08:12:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:35.959013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bfe7b-2c1d-4e54-b623-9edd1b9e2370 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:36.001680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fbcd85-da98-470c-8e34-62354bb738d2 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:36.043357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e3947-4c05-467b-8ee1-2fd30e79e9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:36.044438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e3947-4c05-467b-8ee1-2fd30e79e9dd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:36.044930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e3947-4c05-467b-8ee1-2fd30e79e9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:36.045514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e3947-4c05-467b-8ee1-2fd30e79e9dd None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:36.046705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e3947-4c05-467b-8ee1-2fd30e79e9dd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:36.047227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e3947-4c05-467b-8ee1-2fd30e79e9dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:36.049035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e3947-4c05-467b-8ee1-2fd30e79e9dd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:36.059710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 446/892] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:12:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:36.060992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e3947-4c05-467b-8ee1-2fd30e79e9dd None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:36.080793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bfe7b-2c1d-4e54-b623-9edd1b9e2370 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:36.114221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:36.123901 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe54c88-96fc-42cb-b411-961d0d263499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:36.124816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe54c88-96fc-42cb-b411-961d0d263499 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:36.125223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe54c88-96fc-42cb-b411-961d0d263499 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:36.126043 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe54c88-96fc-42cb-b411-961d0d263499 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:36.127115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe54c88-96fc-42cb-b411-961d0d263499 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:36.127346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe54c88-96fc-42cb-b411-961d0d263499 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:36.128659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:36.133361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe54c88-96fc-42cb-b411-961d0d263499 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:36.160480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 447/893] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Jul 20 08:12:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:36.166367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe54c88-96fc-42cb-b411-961d0d263499 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:36.198909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e3947-4c05-467b-8ee1-2fd30e79e9dd None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:36.238413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f91293-1d90-4e52-8f10-3787392b5cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:36.239489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f91293-1d90-4e52-8f10-3787392b5cad None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:36.239974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f91293-1d90-4e52-8f10-3787392b5cad None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:36.240444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f91293-1d90-4e52-8f10-3787392b5cad None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:36.241403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f91293-1d90-4e52-8f10-3787392b5cad None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:36.241922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f91293-1d90-4e52-8f10-3787392b5cad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:36.243544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f91293-1d90-4e52-8f10-3787392b5cad None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:36.249638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 447/894] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:12:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:36.255979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f91293-1d90-4e52-8f10-3787392b5cad None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:36.293587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe54c88-96fc-42cb-b411-961d0d263499 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:36.333129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8caef7-2879-4a80-9852-93f349c99a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:36.333856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8caef7-2879-4a80-9852-93f349c99a3d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:36.334142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8caef7-2879-4a80-9852-93f349c99a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:36.334422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8caef7-2879-4a80-9852-93f349c99a3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:36.374376 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f91293-1d90-4e52-8f10-3787392b5cad None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:36.390000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:36.402947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:36.415150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922ce95-967b-4cba-8ef2-2912125b3021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:36.415363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8caef7-2879-4a80-9852-93f349c99a3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:36.416181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ce95-967b-4cba-8ef2-2912125b3021 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:36.416794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ce95-967b-4cba-8ef2-2912125b3021 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:36.417215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e922ce95-967b-4cba-8ef2-2912125b3021 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:36.417919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8caef7-2879-4a80-9852-93f349c99a3d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:36.418317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922ce95-967b-4cba-8ef2-2912125b3021 None admin] RBAC: Adding query filter params (interface=public, service_id=bc0d80322e6c45c093c3806d967e3d8a, region_id=RegionOne) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:36.418800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922ce95-967b-4cba-8ef2-2912125b3021 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:36.420326 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e922ce95-967b-4cba-8ef2-2912125b3021 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:36.428136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 448/895] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Jul 20 08:12:36 2022] GET /identity/v3/endpoints?service_id=bc0d80322e6c45c093c3806d967e3d8a&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:36.433351 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922ce95-967b-4cba-8ef2-2912125b3021 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:36.558817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 448/896] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:36 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:12:36.568436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8caef7-2879-4a80-9852-93f349c99a3d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:36.583301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e922ce95-967b-4cba-8ef2-2912125b3021 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:36.631041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338af208-aab0-4cfc-89df-acf93d9b7fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:36.632378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338af208-aab0-4cfc-89df-acf93d9b7fea None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:36.632866 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338af208-aab0-4cfc-89df-acf93d9b7fea None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:36.633521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338af208-aab0-4cfc-89df-acf93d9b7fea None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:36.635047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338af208-aab0-4cfc-89df-acf93d9b7fea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:36.637085 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338af208-aab0-4cfc-89df-acf93d9b7fea None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:36.652927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 449/897] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Jul 20 08:12:36 2022] GET /identity/v3/services => generated 1268 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:12:36.707031 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:36.734705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8caef7-2879-4a80-9852-93f349c99a3d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:36.795155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b041cd-6bff-4c3f-a034-73f5c8b11bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:36.796450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b041cd-6bff-4c3f-a034-73f5c8b11bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:36.796910 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b041cd-6bff-4c3f-a034-73f5c8b11bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:36.798044 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b041cd-6bff-4c3f-a034-73f5c8b11bf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:36.816293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b041cd-6bff-4c3f-a034-73f5c8b11bf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:36.817765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b041cd-6bff-4c3f-a034-73f5c8b11bf0 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-b9b041cd-6bff-4c3f-a034-73f5c8b11bf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:36.838100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:36.839081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:36.840108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:36.840108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:36.840108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:36.840108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:36.844880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 449/898] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:36.846418 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338af208-aab0-4cfc-89df-acf93d9b7fea None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:36.905130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:36.982039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338af208-aab0-4cfc-89df-acf93d9b7fea None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:37.021204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12c16e7-5746-4ced-a5ae-e5c528c58146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:37.021952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c16e7-5746-4ced-a5ae-e5c528c58146 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c16e7-5746-4ced-a5ae-e5c528c58146 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12c16e7-5746-4ced-a5ae-e5c528c58146 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.023321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c16e7-5746-4ced-a5ae-e5c528c58146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:37.023641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c16e7-5746-4ced-a5ae-e5c528c58146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:37.025043 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12c16e7-5746-4ced-a5ae-e5c528c58146 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:37.039416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 450/899] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:37.044944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b041cd-6bff-4c3f-a034-73f5c8b11bf0 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:37.092491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdaa1e-b857-4bc2-b849-c44d5296755d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.092697 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdaa1e-b857-4bc2-b849-c44d5296755d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.092969 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdaa1e-b857-4bc2-b849-c44d5296755d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.094420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 451/900] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:12:37.100445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859c9bd-2916-46ab-89e5-6a7308a1ba79 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.100736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859c9bd-2916-46ab-89e5-6a7308a1ba79 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.101013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859c9bd-2916-46ab-89e5-6a7308a1ba79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.120335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4859c9bd-2916-46ab-89e5-6a7308a1ba79 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:37.122019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4859c9bd-2916-46ab-89e5-6a7308a1ba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:37.175458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b041cd-6bff-4c3f-a034-73f5c8b11bf0 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:37.218890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 452/901] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:37.221296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd6fe2-5ee0-421d-a4bf-c92917e1ce93 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.221593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd6fe2-5ee0-421d-a4bf-c92917e1ce93 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.221763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee7079-0efa-4fc3-8595-61c1fe9ff66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:37.221975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd6fe2-5ee0-421d-a4bf-c92917e1ce93 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.222608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee7079-0efa-4fc3-8595-61c1fe9ff66a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.222888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee7079-0efa-4fc3-8595-61c1fe9ff66a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.223165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee7079-0efa-4fc3-8595-61c1fe9ff66a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.226753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 453/902] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:37.230610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63906783-6a3e-4ab1-af02-a9eac85f39dd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.230610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63906783-6a3e-4ab1-af02-a9eac85f39dd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.230610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63906783-6a3e-4ab1-af02-a9eac85f39dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.237718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee7079-0efa-4fc3-8595-61c1fe9ff66a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:37.240375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee7079-0efa-4fc3-8595-61c1fe9ff66a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:37.242771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:37.251345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-63906783-6a3e-4ab1-af02-a9eac85f39dd None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:37.252581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-63906783-6a3e-4ab1-af02-a9eac85f39dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d0ee7079-0efa-4fc3-8595-61c1fe9ff66a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:37.256942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:37.258142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:37.259623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:37.259623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:37.259623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:37.259623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:37.259623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:37.259623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:37.259623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:37.259623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:37.267700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 450/903] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:37.267700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee7079-0efa-4fc3-8595-61c1fe9ff66a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:37.291823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:37.343775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 454/904] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:37.355671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63906783-6a3e-4ab1-af02-a9eac85f39dd None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:37.401572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee7079-0efa-4fc3-8595-61c1fe9ff66a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:37.452472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739635f7-8c78-4afa-b333-89fbf7a564e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:37.453216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739635f7-8c78-4afa-b333-89fbf7a564e9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.454062 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739635f7-8c78-4afa-b333-89fbf7a564e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.454475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739635f7-8c78-4afa-b333-89fbf7a564e9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.455530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739635f7-8c78-4afa-b333-89fbf7a564e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:37.456924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739635f7-8c78-4afa-b333-89fbf7a564e9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-739635f7-8c78-4afa-b333-89fbf7a564e9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:37.470030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:37.471389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:37.472258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 451/905] 10.222.0.64 () {56 vars in 1132 bytes} [Wed Jul 20 08:12:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:37.473090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739635f7-8c78-4afa-b333-89fbf7a564e9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:37.478512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63906783-6a3e-4ab1-af02-a9eac85f39dd None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:37.518504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffd032-ffd0-4049-b921-df9b989ac75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:37.519336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd032-ffd0-4049-b921-df9b989ac75f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.519625 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd032-ffd0-4049-b921-df9b989ac75f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.519901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ffd032-ffd0-4049-b921-df9b989ac75f None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.520702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffd032-ffd0-4049-b921-df9b989ac75f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:37.521015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffd032-ffd0-4049-b921-df9b989ac75f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:37.522633 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ffd032-ffd0-4049-b921-df9b989ac75f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:37.539039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 455/906] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:37.543792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:37.544955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffd032-ffd0-4049-b921-df9b989ac75f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:37.572204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:37.592077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739635f7-8c78-4afa-b333-89fbf7a564e9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:37.630655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bde4e-186b-4949-9e22-8e775a7f3735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:37.631248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bde4e-186b-4949-9e22-8e775a7f3735 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.631874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bde4e-186b-4949-9e22-8e775a7f3735 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.631874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bde4e-186b-4949-9e22-8e775a7f3735 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.632571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bde4e-186b-4949-9e22-8e775a7f3735 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:37.634075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bde4e-186b-4949-9e22-8e775a7f3735 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:37.649135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 452/907] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:12:37 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:37.650275 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bde4e-186b-4949-9e22-8e775a7f3735 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:37.663005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ffd032-ffd0-4049-b921-df9b989ac75f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:37.716275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a74631-788a-47a5-9139-d42130dba0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:37.717320 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74631-788a-47a5-9139-d42130dba0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.717753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74631-788a-47a5-9139-d42130dba0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.718176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74631-788a-47a5-9139-d42130dba0f6 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.719757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a74631-788a-47a5-9139-d42130dba0f6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:37.720510 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a74631-788a-47a5-9139-d42130dba0f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:37.722330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a74631-788a-47a5-9139-d42130dba0f6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:37.733252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 456/908] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Jul 20 08:12:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:37.734530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a74631-788a-47a5-9139-d42130dba0f6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:37.812470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bde4e-186b-4949-9e22-8e775a7f3735 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:37.841483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:37.851407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:37.859615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560be41-8e4c-453e-959a-782a652216a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:37.861281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560be41-8e4c-453e-959a-782a652216a7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.861743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560be41-8e4c-453e-959a-782a652216a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.862449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560be41-8e4c-453e-959a-782a652216a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.863339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560be41-8e4c-453e-959a-782a652216a7 None admin] RBAC: Adding query filter params (role.id=6747431cfb4944ceadd7cb581c7a23f8, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=5284fc12641540c3a42f291138b7c069) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:37.863660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560be41-8e4c-453e-959a-782a652216a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:37.865710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560be41-8e4c-453e-959a-782a652216a7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:37.872496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 453/909] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:37 2022] GET /identity/v3/role_assignments?user.id=5284fc12641540c3a42f291138b7c069&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=6747431cfb4944ceadd7cb581c7a23f8 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:37.877066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a74631-788a-47a5-9139-d42130dba0f6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:37.922128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99ad56-98e3-43b3-a75a-4d5146009644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:37.923506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99ad56-98e3-43b3-a75a-4d5146009644 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:37.923987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99ad56-98e3-43b3-a75a-4d5146009644 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:37.924327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99ad56-98e3-43b3-a75a-4d5146009644 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:37.925101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99ad56-98e3-43b3-a75a-4d5146009644 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:37.925414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99ad56-98e3-43b3-a75a-4d5146009644 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:37.927171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99ad56-98e3-43b3-a75a-4d5146009644 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:37.933715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 457/910] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Jul 20 08:12:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:37.940641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560be41-8e4c-453e-959a-782a652216a7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:38.064877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560be41-8e4c-453e-959a-782a652216a7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:38.104553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffe285-7867-47d2-8fbc-94f755494f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:38.105648 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffe285-7867-47d2-8fbc-94f755494f27 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:38.106162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffe285-7867-47d2-8fbc-94f755494f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:38.106735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffe285-7867-47d2-8fbc-94f755494f27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:38.107727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffe285-7867-47d2-8fbc-94f755494f27 None admin] RBAC: Adding query filter params (interface=public, service_id=6dd925d2f00242abbc199b17753b927d, region_id=RegionOne) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:38.108236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffe285-7867-47d2-8fbc-94f755494f27 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:38.110077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffe285-7867-47d2-8fbc-94f755494f27 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:38.116860 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 454/911] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Jul 20 08:12:37 2022] GET /identity/v3/endpoints?service_id=6dd925d2f00242abbc199b17753b927d&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:38.122772 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99ad56-98e3-43b3-a75a-4d5146009644 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:38.238442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99ad56-98e3-43b3-a75a-4d5146009644 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:38.276128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd011674-3c4c-4117-a1e3-935a9adc27e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:38.276813 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011674-3c4c-4117-a1e3-935a9adc27e2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:38.277096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011674-3c4c-4117-a1e3-935a9adc27e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:38.277377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd011674-3c4c-4117-a1e3-935a9adc27e2 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:38.278068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd011674-3c4c-4117-a1e3-935a9adc27e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:38.279476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd011674-3c4c-4117-a1e3-935a9adc27e2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:38.286527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 458/912] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Jul 20 08:12:38 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:12:38.938874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754227fd-f74c-479c-b52b-dab5a08a5228 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:38.940113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754227fd-f74c-479c-b52b-dab5a08a5228 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:38.940602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754227fd-f74c-479c-b52b-dab5a08a5228 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:38.942445 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 455/913] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:38.947645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea27b5-e68b-4115-8d47-f9266c36c0c6 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:38.947996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea27b5-e68b-4115-8d47-f9266c36c0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:38.948283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea27b5-e68b-4115-8d47-f9266c36c0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:38.969374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-daea27b5-e68b-4115-8d47-f9266c36c0c6 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:38.971540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-daea27b5-e68b-4115-8d47-f9266c36c0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:38.977072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:39.113741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 459/914] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:39.136405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754227fd-f74c-479c-b52b-dab5a08a5228 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:39.191764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:39.250745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754227fd-f74c-479c-b52b-dab5a08a5228 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:39.287252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435af94a-4c2e-4d23-915b-dff5ac148223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:39.288391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435af94a-4c2e-4d23-915b-dff5ac148223 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:39.288842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435af94a-4c2e-4d23-915b-dff5ac148223 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:39.289257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435af94a-4c2e-4d23-915b-dff5ac148223 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:39.290050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435af94a-4c2e-4d23-915b-dff5ac148223 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:39.291612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435af94a-4c2e-4d23-915b-dff5ac148223 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-435af94a-4c2e-4d23-915b-dff5ac148223 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:39.305191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:39.306408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 08:12:39.307453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:39.309876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 456/915] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:12:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:39.316055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea27b5-e68b-4115-8d47-f9266c36c0c6 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:39.428328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea27b5-e68b-4115-8d47-f9266c36c0c6 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:39.466084 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccc17c-ce18-4d20-a440-28e75fd3d9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:39.466705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc17c-ce18-4d20-a440-28e75fd3d9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:39.466998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc17c-ce18-4d20-a440-28e75fd3d9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:39.467280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc17c-ce18-4d20-a440-28e75fd3d9a7 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:39.468001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccc17c-ce18-4d20-a440-28e75fd3d9a7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:39.468313 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccc17c-ce18-4d20-a440-28e75fd3d9a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:39.470356 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccc17c-ce18-4d20-a440-28e75fd3d9a7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:39.480340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 460/916] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:12:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:39.481753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435af94a-4c2e-4d23-915b-dff5ac148223 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:39.593255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435af94a-4c2e-4d23-915b-dff5ac148223 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:39.609054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:39.632999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c8b20f-5c5b-453e-b7a3-29c88bde6a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:39.634057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8b20f-5c5b-453e-b7a3-29c88bde6a98 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:39.634547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8b20f-5c5b-453e-b7a3-29c88bde6a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:39.635048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8b20f-5c5b-453e-b7a3-29c88bde6a98 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:39.635979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8b20f-5c5b-453e-b7a3-29c88bde6a98 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:39.636496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8b20f-5c5b-453e-b7a3-29c88bde6a98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:39.638087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c8b20f-5c5b-453e-b7a3-29c88bde6a98 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:39.644724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 457/917] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Jul 20 08:12:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:39.654607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccc17c-ce18-4d20-a440-28e75fd3d9a7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:39.655085 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:39.767439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccc17c-ce18-4d20-a440-28e75fd3d9a7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:39.805199 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db7f09-510b-49b8-8bc5-1a2a8550f7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:39.805931 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db7f09-510b-49b8-8bc5-1a2a8550f7a4 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:39.806222 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db7f09-510b-49b8-8bc5-1a2a8550f7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:39.806509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db7f09-510b-49b8-8bc5-1a2a8550f7a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:39.807338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db7f09-510b-49b8-8bc5-1a2a8550f7a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:39.808720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db7f09-510b-49b8-8bc5-1a2a8550f7a4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:39.834760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 461/918] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Jul 20 08:12:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:40.050222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7e41-fbcc-4d6a-aedf-6e2e4326b192 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:40.050840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7e41-fbcc-4d6a-aedf-6e2e4326b192 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:40.050840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7e41-fbcc-4d6a-aedf-6e2e4326b192 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:40.052477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 458/919] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:12:40.057941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d155b46-03b3-495f-88c7-cdeb0d32d957 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:40.058232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d155b46-03b3-495f-88c7-cdeb0d32d957 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:40.058508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d155b46-03b3-495f-88c7-cdeb0d32d957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:40.076373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6d155b46-03b3-495f-88c7-cdeb0d32d957 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:40.077618 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6d155b46-03b3-495f-88c7-cdeb0d32d957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:40.141768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:40.162675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 462/920] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:40.186120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b7e41-fbcc-4d6a-aedf-6e2e4326b192 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:40.315684 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b7e41-fbcc-4d6a-aedf-6e2e4326b192 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:40.358543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9383c21-3da5-433f-91ef-d8d734a25237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:40.359567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9383c21-3da5-433f-91ef-d8d734a25237 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:40.360492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9383c21-3da5-433f-91ef-d8d734a25237 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:40.361180 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9383c21-3da5-433f-91ef-d8d734a25237 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:40.375509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9383c21-3da5-433f-91ef-d8d734a25237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e9383c21-3da5-433f-91ef-d8d734a25237 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:40.385631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:40.386618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:40.387440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 459/921] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Jul 20 08:12:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:40.394680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d155b46-03b3-495f-88c7-cdeb0d32d957 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:40.467114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986e26a-e74f-4a61-8a28-0a6035580e28 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:40.467450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986e26a-e74f-4a61-8a28-0a6035580e28 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:40.467666 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986e26a-e74f-4a61-8a28-0a6035580e28 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:40.469405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 460/922] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:40.475386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22344-81e8-4b5a-87ef-05ee0a66932e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:40.475828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22344-81e8-4b5a-87ef-05ee0a66932e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:40.476119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe22344-81e8-4b5a-87ef-05ee0a66932e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:40.496192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4fe22344-81e8-4b5a-87ef-05ee0a66932e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:40.497981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4fe22344-81e8-4b5a-87ef-05ee0a66932e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:40.514197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d155b46-03b3-495f-88c7-cdeb0d32d957 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:40.537612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:40.559568 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312facf-af6c-4a5c-9e11-cb127eec06ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:40.560323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312facf-af6c-4a5c-9e11-cb127eec06ab None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:40.560946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312facf-af6c-4a5c-9e11-cb127eec06ab None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:40.561368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312facf-af6c-4a5c-9e11-cb127eec06ab None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:40.562389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312facf-af6c-4a5c-9e11-cb127eec06ab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:40.563531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312facf-af6c-4a5c-9e11-cb127eec06ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:40.565445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312facf-af6c-4a5c-9e11-cb127eec06ab None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:40.579189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 463/923] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Jul 20 08:12:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:40.584692 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312facf-af6c-4a5c-9e11-cb127eec06ab None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:40.599148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 461/924] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:40.623596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe22344-81e8-4b5a-87ef-05ee0a66932e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:40.678615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:40.760882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312facf-af6c-4a5c-9e11-cb127eec06ab None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:40.763758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe22344-81e8-4b5a-87ef-05ee0a66932e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:40.804793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a8e1-a4ac-406f-9661-66f9ab571517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:40.805614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a8e1-a4ac-406f-9661-66f9ab571517 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:40.805903 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a8e1-a4ac-406f-9661-66f9ab571517 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:40.806049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b4dda-9f12-4c14-8679-8a3b34e107e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:40.806195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a8e1-a4ac-406f-9661-66f9ab571517 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:40.806786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4dda-9f12-4c14-8679-8a3b34e107e1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:40.806939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162a8e1-a4ac-406f-9661-66f9ab571517 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:40.807061 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4dda-9f12-4c14-8679-8a3b34e107e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:40.807343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4dda-9f12-4c14-8679-8a3b34e107e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:40.808235 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8162a8e1-a4ac-406f-9661-66f9ab571517 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:40.819426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b4dda-9f12-4c14-8679-8a3b34e107e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:40.820889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b4dda-9f12-4c14-8679-8a3b34e107e1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-8162a8e1-a4ac-406f-9661-66f9ab571517 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:40.822707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:40.823703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:40.824565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 462/925] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Jul 20 08:12:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:40.825368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d2d18-e16c-4939-899f-dc88534bff4d None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:40.825437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d2d18-e16c-4939-899f-dc88534bff4d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:40.825851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d2d18-e16c-4939-899f-dc88534bff4d None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:40.827147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 463/926] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:40.834947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d2d18-e16c-4939-899f-dc88534bff4d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-0d7b4dda-9f12-4c14-8679-8a3b34e107e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:40.849941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:40.851155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:40.852067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:40.852067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:40.852067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:40.852067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:40.852067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 464/927] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:40.853386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cd1b7-861a-487f-96c4-32c773811e1a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:40.853749 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cd1b7-861a-487f-96c4-32c773811e1a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:40.854105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9cd1b7-861a-487f-96c4-32c773811e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:40.871696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ef9cd1b7-861a-487f-96c4-32c773811e1a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:40.872923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ef9cd1b7-861a-487f-96c4-32c773811e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:40.880380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:40.950783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 465/928] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:40.952068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cd1b7-861a-487f-96c4-32c773811e1a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:40.978443 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d2d18-e16c-4939-899f-dc88534bff4d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:40.984485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:41.022877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04298-68ed-4b82-b8df-a42f3af1ccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:41.023025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04298-68ed-4b82-b8df-a42f3af1ccd6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:41.023261 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04298-68ed-4b82-b8df-a42f3af1ccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:41.023540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04298-68ed-4b82-b8df-a42f3af1ccd6 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:41.024255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea04298-68ed-4b82-b8df-a42f3af1ccd6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:41.024565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea04298-68ed-4b82-b8df-a42f3af1ccd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:41.026220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea04298-68ed-4b82-b8df-a42f3af1ccd6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:41.037153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 464/929] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Jul 20 08:12:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:41.037153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04298-68ed-4b82-b8df-a42f3af1ccd6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:41.082275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9cd1b7-861a-487f-96c4-32c773811e1a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:41.121309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa3da8-2867-43e9-ad3f-cf54e91c8488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:41.121896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa3da8-2867-43e9-ad3f-cf54e91c8488 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:41.122179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa3da8-2867-43e9-ad3f-cf54e91c8488 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:41.122455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa3da8-2867-43e9-ad3f-cf54e91c8488 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:41.123216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa3da8-2867-43e9-ad3f-cf54e91c8488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:41.123523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa3da8-2867-43e9-ad3f-cf54e91c8488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:41.124902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa3da8-2867-43e9-ad3f-cf54e91c8488 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:41.146331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 466/930] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:41.147212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa3da8-2867-43e9-ad3f-cf54e91c8488 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:41.160804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:41.167876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04298-68ed-4b82-b8df-a42f3af1ccd6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:41.213426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06cff9-d2d7-4094-bc27-c480f410a756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:41.214236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06cff9-d2d7-4094-bc27-c480f410a756 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:41.214517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06cff9-d2d7-4094-bc27-c480f410a756 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:41.214971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06cff9-d2d7-4094-bc27-c480f410a756 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:41.215470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06cff9-d2d7-4094-bc27-c480f410a756 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:41.216852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06cff9-d2d7-4094-bc27-c480f410a756 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ed06cff9-d2d7-4094-bc27-c480f410a756 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:41.230068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:41.231158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:41.237361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 465/931] 10.222.0.64 () {56 vars in 1132 bytes} [Wed Jul 20 08:12:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:41.238341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06cff9-d2d7-4094-bc27-c480f410a756 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:41.257418 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:41.268319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa3da8-2867-43e9-ad3f-cf54e91c8488 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:41.303680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83407dc4-2928-42cc-98d8-0f4c631b3908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:41.304354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83407dc4-2928-42cc-98d8-0f4c631b3908 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:41.304644 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83407dc4-2928-42cc-98d8-0f4c631b3908 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:41.304927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83407dc4-2928-42cc-98d8-0f4c631b3908 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:41.305643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83407dc4-2928-42cc-98d8-0f4c631b3908 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:41.305956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83407dc4-2928-42cc-98d8-0f4c631b3908 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:41.307367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83407dc4-2928-42cc-98d8-0f4c631b3908 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:41.312376 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 467/932] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Jul 20 08:12:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:41.313911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83407dc4-2928-42cc-98d8-0f4c631b3908 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:41.365072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06cff9-d2d7-4094-bc27-c480f410a756 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:41.403907 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62796764-56ee-40a2-a1c3-bf0ca7fbfaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:41.405017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62796764-56ee-40a2-a1c3-bf0ca7fbfaac None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:41.405371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62796764-56ee-40a2-a1c3-bf0ca7fbfaac None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:41.405700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62796764-56ee-40a2-a1c3-bf0ca7fbfaac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:41.415202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62796764-56ee-40a2-a1c3-bf0ca7fbfaac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:41.416693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62796764-56ee-40a2-a1c3-bf0ca7fbfaac None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:41.426035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83407dc4-2928-42cc-98d8-0f4c631b3908 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-62796764-56ee-40a2-a1c3-bf0ca7fbfaac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:41.432660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:41.433652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:41.434792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:41.434792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:41.434792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:41.434792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:41.434792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:41.434792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:41.434792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:41.434792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:41.434792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:41.434792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:41.434792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 466/933] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Jul 20 08:12:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:41.435365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62796764-56ee-40a2-a1c3-bf0ca7fbfaac None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:41.464529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:41.471250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aed651-4aab-414a-b737-3106dab853da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:41.471927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aed651-4aab-414a-b737-3106dab853da None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:41.472218 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aed651-4aab-414a-b737-3106dab853da None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:41.472508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aed651-4aab-414a-b737-3106dab853da None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:41.473224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aed651-4aab-414a-b737-3106dab853da None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:41.473554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aed651-4aab-414a-b737-3106dab853da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:41.474954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aed651-4aab-414a-b737-3106dab853da None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:41.480309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 468/934] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Jul 20 08:12:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:41.480952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aed651-4aab-414a-b737-3106dab853da None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:41.504176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:41.562364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62796764-56ee-40a2-a1c3-bf0ca7fbfaac None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:41.599322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aed651-4aab-414a-b737-3106dab853da None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:41.612310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19237f4b-6d3b-496f-8b2e-8d837ebc4674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:41.613279 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19237f4b-6d3b-496f-8b2e-8d837ebc4674 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:41.613723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19237f4b-6d3b-496f-8b2e-8d837ebc4674 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:41.614121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19237f4b-6d3b-496f-8b2e-8d837ebc4674 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:41.615260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19237f4b-6d3b-496f-8b2e-8d837ebc4674 None admin] RBAC: Adding query filter params (interface=public, service_id=ae267e0663b54eb2871df96b3e06877a, region_id=RegionOne) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:41.615797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19237f4b-6d3b-496f-8b2e-8d837ebc4674 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:41.617518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19237f4b-6d3b-496f-8b2e-8d837ebc4674 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:41.624012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 467/935] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Jul 20 08:12:41 2022] GET /identity/v3/endpoints?service_id=ae267e0663b54eb2871df96b3e06877a&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:41.625977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19237f4b-6d3b-496f-8b2e-8d837ebc4674 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:41.647496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8486f-019b-49af-b7b1-fa47730791d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:41.648507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8486f-019b-49af-b7b1-fa47730791d6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:41.648989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8486f-019b-49af-b7b1-fa47730791d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:41.649265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8486f-019b-49af-b7b1-fa47730791d6 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:41.650205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8486f-019b-49af-b7b1-fa47730791d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:41.650522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8486f-019b-49af-b7b1-fa47730791d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:41.651998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8486f-019b-49af-b7b1-fa47730791d6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:41.671102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 469/936] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Jul 20 08:12:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:41.671861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8486f-019b-49af-b7b1-fa47730791d6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:41.733892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:41.756025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19237f4b-6d3b-496f-8b2e-8d837ebc4674 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:41.766149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:41.795517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8486f-019b-49af-b7b1-fa47730791d6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:41.802015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0677707-dd93-4e02-93a9-d4afdc2d99f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:41.802717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0677707-dd93-4e02-93a9-d4afdc2d99f5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:41.802999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0677707-dd93-4e02-93a9-d4afdc2d99f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:41.803272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0677707-dd93-4e02-93a9-d4afdc2d99f5 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:41.803990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0677707-dd93-4e02-93a9-d4afdc2d99f5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:41.804301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0677707-dd93-4e02-93a9-d4afdc2d99f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:41.805688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0677707-dd93-4e02-93a9-d4afdc2d99f5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:41.816688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 468/937] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Jul 20 08:12:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:41.816688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0677707-dd93-4e02-93a9-d4afdc2d99f5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:41.840817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d50ec-c089-4d4a-bebf-39875402b6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:41.841544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d50ec-c089-4d4a-bebf-39875402b6fa None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:41.841832 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d50ec-c089-4d4a-bebf-39875402b6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:41.842111 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d50ec-c089-4d4a-bebf-39875402b6fa None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:41.842854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d50ec-c089-4d4a-bebf-39875402b6fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:41.844341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d50ec-c089-4d4a-bebf-39875402b6fa None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:41.862621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 470/938] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Jul 20 08:12:41 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:12:41.862621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d50ec-c089-4d4a-bebf-39875402b6fa None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:41.975456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0677707-dd93-4e02-93a9-d4afdc2d99f5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:41.997154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d50ec-c089-4d4a-bebf-39875402b6fa None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:42.043911 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb023a5b-f346-46b3-972d-fe130b423690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:42.045253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb023a5b-f346-46b3-972d-fe130b423690 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:42.045765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb023a5b-f346-46b3-972d-fe130b423690 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:42.046047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb023a5b-f346-46b3-972d-fe130b423690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:42.057335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de08d4-e076-43e5-abbe-163cb8cefe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:42.058663 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de08d4-e076-43e5-abbe-163cb8cefe83 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:42.058957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de08d4-e076-43e5-abbe-163cb8cefe83 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:42.059243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de08d4-e076-43e5-abbe-163cb8cefe83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:42.060013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de08d4-e076-43e5-abbe-163cb8cefe83 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:42.061075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb023a5b-f346-46b3-972d-fe130b423690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:42.061407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de08d4-e076-43e5-abbe-163cb8cefe83 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:42.069634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb023a5b-f346-46b3-972d-fe130b423690 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:42.072635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:42.078522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-bb023a5b-f346-46b3-972d-fe130b423690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:42.085109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:42.086117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:42.087051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:42.087051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:42.087051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:42.087051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:42.088845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 469/939] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:42.090306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0d8e7-4576-43b8-ab7e-0dbc3632ac84 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:42.090624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0d8e7-4576-43b8-ab7e-0dbc3632ac84 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:42.090966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0d8e7-4576-43b8-ab7e-0dbc3632ac84 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:42.093037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 471/940] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Jul 20 08:12:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:42.096495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de08d4-e076-43e5-abbe-163cb8cefe83 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:42.096944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 470/941] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:42.098908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80565db-afe5-402b-aa7c-4f66a3f05f57 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:42.099206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80565db-afe5-402b-aa7c-4f66a3f05f57 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:42.099488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80565db-afe5-402b-aa7c-4f66a3f05f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:42.128205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e80565db-afe5-402b-aa7c-4f66a3f05f57 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:42.130436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e80565db-afe5-402b-aa7c-4f66a3f05f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:42.244586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 471/942] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:42.247397 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de08d4-e076-43e5-abbe-163cb8cefe83 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:42.271362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80565db-afe5-402b-aa7c-4f66a3f05f57 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:42.307467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9ef24-0155-4819-8e9d-6abfb9ba9d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:42.308153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9ef24-0155-4819-8e9d-6abfb9ba9d0b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:42.308441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9ef24-0155-4819-8e9d-6abfb9ba9d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:42.308723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad9ef24-0155-4819-8e9d-6abfb9ba9d0b None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:42.309524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9ef24-0155-4819-8e9d-6abfb9ba9d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:42.309847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9ef24-0155-4819-8e9d-6abfb9ba9d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:42.311286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad9ef24-0155-4819-8e9d-6abfb9ba9d0b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:42.325963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 472/943] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:42.331780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9ef24-0155-4819-8e9d-6abfb9ba9d0b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:42.419072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80565db-afe5-402b-aa7c-4f66a3f05f57 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:42.425414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:42.456332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:42.467275 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c3066-264c-4b13-917b-d99039fa0c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:42.467947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c3066-264c-4b13-917b-d99039fa0c33 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:42.468234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c3066-264c-4b13-917b-d99039fa0c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:42.468515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c3066-264c-4b13-917b-d99039fa0c33 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:42.470080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c3066-264c-4b13-917b-d99039fa0c33 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:42.472087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c3066-264c-4b13-917b-d99039fa0c33 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:42.473515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: WARNING keystone.server.flask.common [None req-d47c3066-264c-4b13-917b-d99039fa0c33 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 08:12:42.492209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad9ef24-0155-4819-8e9d-6abfb9ba9d0b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:42.498196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.sql.core [None req-d47c3066-264c-4b13-917b-d99039fa0c33 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 08:12:42.498196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 08:12:42.498196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [parameters: {'id': '75b4aab0d28141789246c74a905c1cad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 08:12:42.498196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80226) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d47c3066-264c-4b13-917b-d99039fa0c33 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 75b4aab0d28141789246c74a905c1cad.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 75b4aab0d28141789246c74a905c1cad. Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application conn.query(q) Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 08:12:42.519102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result.read() Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 08:12:42.519990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application next(self.gen) Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application yield resource Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application next(self.gen) Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise self.value Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application session.commit() Jul 20 08:12:42.520972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.session.flush() Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application compat.raise_( Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 08:12:42.522470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 08:12:42.523370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application conn.query(q) Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result.read() Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 08:12:42.525537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [parameters: {'id': '75b4aab0d28141789246c74a905c1cad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:42.527830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:42.529905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:42.533757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:42.533757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 08:12:42.533757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 08:12:42.533757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:42.533757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:42.533757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 08:12:42.533757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 08:12:42.533757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 75b4aab0d28141789246c74a905c1cad. Jul 20 08:12:42.533757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:42.533757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 472/944] 10.222.0.64 () {60 vars in 1140 bytes} [Wed Jul 20 08:12:42 2022] POST /identity/v3/projects => generated 262 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:42.533757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c3066-264c-4b13-917b-d99039fa0c33 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:42.555637 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bf73f-a4f1-4652-96ad-9946bd512152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:42.557474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bf73f-a4f1-4652-96ad-9946bd512152 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:42.558129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bf73f-a4f1-4652-96ad-9946bd512152 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:42.558810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bf73f-a4f1-4652-96ad-9946bd512152 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:42.576174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bf73f-a4f1-4652-96ad-9946bd512152 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:42.578380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bf73f-a4f1-4652-96ad-9946bd512152 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-028bf73f-a4f1-4652-96ad-9946bd512152 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:42.595421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:42.596409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:42.597343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:42.597343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:42.597343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:42.597343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:42.597343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:42.597343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:42.597343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:42.597343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:42.600009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 473/945] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:42.606009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bf73f-a4f1-4652-96ad-9946bd512152 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:42.668715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c3066-264c-4b13-917b-d99039fa0c33 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:42.712118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5479109-5332-466b-b5f5-f8f0737b689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:42.713471 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5479109-5332-466b-b5f5-f8f0737b689f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:42.713966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5479109-5332-466b-b5f5-f8f0737b689f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:42.714449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5479109-5332-466b-b5f5-f8f0737b689f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:42.723298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5479109-5332-466b-b5f5-f8f0737b689f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:42.725310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5479109-5332-466b-b5f5-f8f0737b689f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:42.735609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bf73f-a4f1-4652-96ad-9946bd512152 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-a5479109-5332-466b-b5f5-f8f0737b689f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:42.742534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:42.743470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:42.744352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:42.744352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:42.744352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:42.744352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:42.744352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:42.744352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:42.744352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:42.744352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:42.747050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 473/946] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:42.754800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5479109-5332-466b-b5f5-f8f0737b689f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:42.757657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:42.775193 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff0067-367d-4966-a225-641347bc9680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:42.775865 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff0067-367d-4966-a225-641347bc9680 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:42.776151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff0067-367d-4966-a225-641347bc9680 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:42.776430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff0067-367d-4966-a225-641347bc9680 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:42.777361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff0067-367d-4966-a225-641347bc9680 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:42.777733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff0067-367d-4966-a225-641347bc9680 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:42.779397 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ff0067-367d-4966-a225-641347bc9680 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:42.794687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 474/947] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:42.809485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff0067-367d-4966-a225-641347bc9680 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:42.815638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:42.888977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5479109-5332-466b-b5f5-f8f0737b689f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:42.929926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306dc9bf-9784-45ad-b2e9-200127232a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:42.930604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dc9bf-9784-45ad-b2e9-200127232a33 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:42.930899 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dc9bf-9784-45ad-b2e9-200127232a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:42.931173 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306dc9bf-9784-45ad-b2e9-200127232a33 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:42.932007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306dc9bf-9784-45ad-b2e9-200127232a33 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:42.932312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306dc9bf-9784-45ad-b2e9-200127232a33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:42.933842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306dc9bf-9784-45ad-b2e9-200127232a33 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:42.935404 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ff0067-367d-4966-a225-641347bc9680 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:42.957731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 474/948] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:42.978666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c515b80f-2231-45ff-bf40-2bdbf2464e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:42.979651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515b80f-2231-45ff-bf40-2bdbf2464e51 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:42.979987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515b80f-2231-45ff-bf40-2bdbf2464e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:42.980269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c515b80f-2231-45ff-bf40-2bdbf2464e51 None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/5284fc12641540c3a42f291138b7c069/roles/6747431cfb4944ceadd7cb581c7a23f8` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:42.982801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515b80f-2231-45ff-bf40-2bdbf2464e51 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=5284fc12641540c3a42f291138b7c069, role_id=6747431cfb4944ceadd7cb581c7a23f8)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:42.984313 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c515b80f-2231-45ff-bf40-2bdbf2464e51 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:43.006177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 475/949] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:12:42 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/5284fc12641540c3a42f291138b7c069/roles/6747431cfb4944ceadd7cb581c7a23f8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:12:44.075353 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56dc18-5e85-4171-b8ae-9844b63f48bc None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:44.076020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56dc18-5e85-4171-b8ae-9844b63f48bc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:44.076020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56dc18-5e85-4171-b8ae-9844b63f48bc None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:44.077409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 475/950] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:44.083551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef8d59-d55a-4176-943d-2b6cd77b3357 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:44.084384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef8d59-d55a-4176-943d-2b6cd77b3357 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:44.084879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef8d59-d55a-4176-943d-2b6cd77b3357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:44.102848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7cef8d59-d55a-4176-943d-2b6cd77b3357 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:44.104380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7cef8d59-d55a-4176-943d-2b6cd77b3357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:44.135728 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:44.183330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 476/951] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:44.207421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56dc18-5e85-4171-b8ae-9844b63f48bc None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:44.307785 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb1460-0435-47ab-812a-bb17b6531064 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:44.308628 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb1460-0435-47ab-812a-bb17b6531064 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:44.309116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb1460-0435-47ab-812a-bb17b6531064 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:44.311214 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 477/952] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:12:44.317276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f032c-8272-4365-905b-faecdf5b9701 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:44.317622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f032c-8272-4365-905b-faecdf5b9701 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:44.317848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f032c-8272-4365-905b-faecdf5b9701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:44.336152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-436f032c-8272-4365-905b-faecdf5b9701 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:44.337661 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-436f032c-8272-4365-905b-faecdf5b9701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:44.341385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56dc18-5e85-4171-b8ae-9844b63f48bc None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:44.343269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:44.383343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fcf53-8874-4717-ab62-74ff8ea66d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:44.384335 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fcf53-8874-4717-ab62-74ff8ea66d33 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:44.384717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fcf53-8874-4717-ab62-74ff8ea66d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:44.385148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fcf53-8874-4717-ab62-74ff8ea66d33 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:44.386037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fcf53-8874-4717-ab62-74ff8ea66d33 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:44.387636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fcf53-8874-4717-ab62-74ff8ea66d33 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5e9fcf53-8874-4717-ab62-74ff8ea66d33 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:44.401939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:44.403348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:44.404515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:44.406444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 476/953] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Jul 20 08:12:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:44.413597 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fcf53-8874-4717-ab62-74ff8ea66d33 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:44.442750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 478/954] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:44.461563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f032c-8272-4365-905b-faecdf5b9701 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:44.535973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fcf53-8874-4717-ab62-74ff8ea66d33 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:44.573233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731afc8-3dd1-4eb3-8406-a0c8a4b80e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:44.573470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:44.574354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731afc8-3dd1-4eb3-8406-a0c8a4b80e36 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:44.574822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731afc8-3dd1-4eb3-8406-a0c8a4b80e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:44.575207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731afc8-3dd1-4eb3-8406-a0c8a4b80e36 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:44.576037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731afc8-3dd1-4eb3-8406-a0c8a4b80e36 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:44.576530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731afc8-3dd1-4eb3-8406-a0c8a4b80e36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:44.578032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731afc8-3dd1-4eb3-8406-a0c8a4b80e36 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:44.583352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f032c-8272-4365-905b-faecdf5b9701 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:44.584598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 477/955] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Jul 20 08:12:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:44.589973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731afc8-3dd1-4eb3-8406-a0c8a4b80e36 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:44.624738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:44.629260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971081ed-cb00-4f15-811b-4c18338d72e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:44.630439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971081ed-cb00-4f15-811b-4c18338d72e6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:44.633162 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971081ed-cb00-4f15-811b-4c18338d72e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:44.633525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971081ed-cb00-4f15-811b-4c18338d72e6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:44.634454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971081ed-cb00-4f15-811b-4c18338d72e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:44.636767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971081ed-cb00-4f15-811b-4c18338d72e6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-971081ed-cb00-4f15-811b-4c18338d72e6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:44.654702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:44.656011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 08:12:44.657246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:44.664117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 479/956] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:44.665176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971081ed-cb00-4f15-811b-4c18338d72e6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:44.723999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731afc8-3dd1-4eb3-8406-a0c8a4b80e36 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:44.766296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dd40f-d515-4f34-ae67-c8819f499b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:44.767417 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dd40f-d515-4f34-ae67-c8819f499b67 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:44.767868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dd40f-d515-4f34-ae67-c8819f499b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:44.768301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dd40f-d515-4f34-ae67-c8819f499b67 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:44.769263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dd40f-d515-4f34-ae67-c8819f499b67 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:44.769719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dd40f-d515-4f34-ae67-c8819f499b67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:44.771501 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3dd40f-d515-4f34-ae67-c8819f499b67 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:44.782252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971081ed-cb00-4f15-811b-4c18338d72e6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:44.783246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 478/957] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Jul 20 08:12:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:44.784320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dd40f-d515-4f34-ae67-c8819f499b67 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:44.817386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66626459-d7c0-4b78-9e2c-887e081ff5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:44.818221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66626459-d7c0-4b78-9e2c-887e081ff5ae None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:44.818811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66626459-d7c0-4b78-9e2c-887e081ff5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:44.818988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66626459-d7c0-4b78-9e2c-887e081ff5ae None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:44.819822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66626459-d7c0-4b78-9e2c-887e081ff5ae None admin] RBAC: Adding query filter params (name=compute) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:44.820237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66626459-d7c0-4b78-9e2c-887e081ff5ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:44.821710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66626459-d7c0-4b78-9e2c-887e081ff5ae None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:44.832247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 480/958] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:44.833244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66626459-d7c0-4b78-9e2c-887e081ff5ae None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:44.860251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:44.898645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:44.908221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dd40f-d515-4f34-ae67-c8819f499b67 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:44.943227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66626459-d7c0-4b78-9e2c-887e081ff5ae None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:44.949847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c22b96-aff9-4a7b-97bc-6b9f352b9730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:44.950741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c22b96-aff9-4a7b-97bc-6b9f352b9730 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:44.951267 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c22b96-aff9-4a7b-97bc-6b9f352b9730 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:44.951705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c22b96-aff9-4a7b-97bc-6b9f352b9730 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:44.952611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c22b96-aff9-4a7b-97bc-6b9f352b9730 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:44.954289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c22b96-aff9-4a7b-97bc-6b9f352b9730 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:44.986757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 479/959] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Jul 20 08:12:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:44.996007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251196b2-b2c0-40dd-bbf3-03c7642b42e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:44.996892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251196b2-b2c0-40dd-bbf3-03c7642b42e6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:44.997287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251196b2-b2c0-40dd-bbf3-03c7642b42e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:44.997668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251196b2-b2c0-40dd-bbf3-03c7642b42e6 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:44.998513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251196b2-b2c0-40dd-bbf3-03c7642b42e6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:44.999008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251196b2-b2c0-40dd-bbf3-03c7642b42e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:45.001301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251196b2-b2c0-40dd-bbf3-03c7642b42e6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:45.007762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 481/960] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:45.234411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19431229-6a6c-41e1-b2fc-5088a1e679f9 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:45.235526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19431229-6a6c-41e1-b2fc-5088a1e679f9 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:45.235605 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19431229-6a6c-41e1-b2fc-5088a1e679f9 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:45.237553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 480/961] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:45.245270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f4b5-35e6-4e9f-bae8-9c212dbf59d2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:45.245270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f4b5-35e6-4e9f-bae8-9c212dbf59d2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:45.245270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65f4b5-35e6-4e9f-bae8-9c212dbf59d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:45.261993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ae65f4b5-35e6-4e9f-bae8-9c212dbf59d2 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:45.263582 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ae65f4b5-35e6-4e9f-bae8-9c212dbf59d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:45.318988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5773ac6-e572-44c0-aba6-4bba7224e55a None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:45.319278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5773ac6-e572-44c0-aba6-4bba7224e55a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:45.319562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5773ac6-e572-44c0-aba6-4bba7224e55a None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:45.321278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 481/962] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:45.326770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e72d9-89cf-4d9c-b8b1-4cf309891963 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:45.327064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e72d9-89cf-4d9c-b8b1-4cf309891963 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:45.327348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e72d9-89cf-4d9c-b8b1-4cf309891963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:45.346704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 482/963] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:45.347142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d87e72d9-89cf-4d9c-b8b1-4cf309891963 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:45.348713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d87e72d9-89cf-4d9c-b8b1-4cf309891963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:45.367973 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65f4b5-35e6-4e9f-bae8-9c212dbf59d2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:45.385124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:45.433833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 482/964] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:45.460301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e72d9-89cf-4d9c-b8b1-4cf309891963 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:45.485434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65f4b5-35e6-4e9f-bae8-9c212dbf59d2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:45.531250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2504f4-020d-4501-8777-138ab9525dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:45.532381 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2504f4-020d-4501-8777-138ab9525dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:45.532670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2504f4-020d-4501-8777-138ab9525dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:45.532949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2504f4-020d-4501-8777-138ab9525dd1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:45.552984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2504f4-020d-4501-8777-138ab9525dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:45.558988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-cb2504f4-020d-4501-8777-138ab9525dd1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:45.564155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:45.565182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:45.571019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 483/965] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Jul 20 08:12:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:45.574873 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2504f4-020d-4501-8777-138ab9525dd1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:45.581130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e72d9-89cf-4d9c-b8b1-4cf309891963 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:45.620394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90180487-11c7-4c45-a23b-d47b12389842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:45.621681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90180487-11c7-4c45-a23b-d47b12389842 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:45.623337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90180487-11c7-4c45-a23b-d47b12389842 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:45.623926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90180487-11c7-4c45-a23b-d47b12389842 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:45.625071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90180487-11c7-4c45-a23b-d47b12389842 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:45.627634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90180487-11c7-4c45-a23b-d47b12389842 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:45.645738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.sql.core [None req-90180487-11c7-4c45-a23b-d47b12389842 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 08:12:45.645738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 08:12:45.645738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [parameters: {'user_id': '2a0e3128faeb40fda4af20f6953b1949', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 08:12:45.645738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80226) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-90180487-11c7-4c45-a23b-d47b12389842 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application conn.query(q) Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result.read() Jul 20 08:12:45.655283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 08:12:45.656193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application next(self.gen) Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application yield resource Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application next(self.gen) Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise self.value Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application session.commit() Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 08:12:45.657274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.session.flush() Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application compat.raise_( Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 08:12:45.658323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 08:12:45.659835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application conn.query(q) Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application result.read() Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 08:12:45.667991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [parameters: {'user_id': '2a0e3128faeb40fda4af20f6953b1949', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:45.670706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 08:12:45.673995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 08:12:45.677135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 08:12:45.677135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 08:12:45.677135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 08:12:45.677135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 08:12:45.677135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 08:12:45.677135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:45.677135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 483/966] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Jul 20 08:12:45 2022] POST /identity/v3/users => generated 160 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:45.677135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90180487-11c7-4c45-a23b-d47b12389842 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:45.692076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:45.730491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2504f4-020d-4501-8777-138ab9525dd1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:45.773731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe525e97-6ce7-4c7b-8eee-c8292a1a240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:45.774419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe525e97-6ce7-4c7b-8eee-c8292a1a240e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:45.774761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe525e97-6ce7-4c7b-8eee-c8292a1a240e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:45.775037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe525e97-6ce7-4c7b-8eee-c8292a1a240e None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:45.776116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe525e97-6ce7-4c7b-8eee-c8292a1a240e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:45.776454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe525e97-6ce7-4c7b-8eee-c8292a1a240e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:45.778186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe525e97-6ce7-4c7b-8eee-c8292a1a240e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:45.794123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 484/967] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Jul 20 08:12:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:45.796146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe525e97-6ce7-4c7b-8eee-c8292a1a240e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:45.828035 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90180487-11c7-4c45-a23b-d47b12389842 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:45.884683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9f23a-afd0-4d7f-a98a-7da80a1bb6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:45.886736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9f23a-afd0-4d7f-a98a-7da80a1bb6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:45.887314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9f23a-afd0-4d7f-a98a-7da80a1bb6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:45.888100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9f23a-afd0-4d7f-a98a-7da80a1bb6d1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:45.903009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9f23a-afd0-4d7f-a98a-7da80a1bb6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:45.905310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9f23a-afd0-4d7f-a98a-7da80a1bb6d1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-4cd9f23a-afd0-4d7f-a98a-7da80a1bb6d1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:45.927017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:45.928130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:45.929075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:45.929075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:45.929075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:45.929075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:45.929075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:45.929075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:45.929075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:45.929075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:45.929075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 08:12:45.929075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:45.937642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 484/968] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:12:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:45.940601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9f23a-afd0-4d7f-a98a-7da80a1bb6d1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:45.945292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:45.950858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe525e97-6ce7-4c7b-8eee-c8292a1a240e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:45.992554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a97fc-e185-4721-9672-104a4332421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:45.993668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a97fc-e185-4721-9672-104a4332421a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:45.993980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a97fc-e185-4721-9672-104a4332421a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:45.994469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a97fc-e185-4721-9672-104a4332421a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:46.001002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:46.022943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a97fc-e185-4721-9672-104a4332421a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:46.024696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a97fc-e185-4721-9672-104a4332421a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d25a97fc-e185-4721-9672-104a4332421a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:46.050566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:46.051876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:46.053327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:46.053327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:46.053327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:46.053327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:46.059150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 485/969] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:46.060554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a97fc-e185-4721-9672-104a4332421a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:46.092621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9f23a-afd0-4d7f-a98a-7da80a1bb6d1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:46.133482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1198dd-f005-4d43-b187-30bf2aaf47dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:46.134441 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1198dd-f005-4d43-b187-30bf2aaf47dc None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:46.134930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1198dd-f005-4d43-b187-30bf2aaf47dc None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:46.135383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1198dd-f005-4d43-b187-30bf2aaf47dc None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:46.136350 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1198dd-f005-4d43-b187-30bf2aaf47dc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:46.136880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1198dd-f005-4d43-b187-30bf2aaf47dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:46.139529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1198dd-f005-4d43-b187-30bf2aaf47dc None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:46.175885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a97fc-e185-4721-9672-104a4332421a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:46.197311 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 485/970] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Jul 20 08:12:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:46.216729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6d78a-2598-4cb4-85ab-8a07d4a40b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:46.217426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6d78a-2598-4cb4-85ab-8a07d4a40b17 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:46.217716 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6d78a-2598-4cb4-85ab-8a07d4a40b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:46.217998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6d78a-2598-4cb4-85ab-8a07d4a40b17 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:46.218806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6d78a-2598-4cb4-85ab-8a07d4a40b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:46.219746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6d78a-2598-4cb4-85ab-8a07d4a40b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:46.221252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6d78a-2598-4cb4-85ab-8a07d4a40b17 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:46.238235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 486/971] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:46.244325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1198dd-f005-4d43-b187-30bf2aaf47dc None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:46.325113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:46.383834 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1198dd-f005-4d43-b187-30bf2aaf47dc None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:46.426875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f53ccd-13d3-46b5-b3de-a2fd59e381e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:46.427962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f53ccd-13d3-46b5-b3de-a2fd59e381e6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:46.428246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f53ccd-13d3-46b5-b3de-a2fd59e381e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:46.428525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f53ccd-13d3-46b5-b3de-a2fd59e381e6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:46.438733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f53ccd-13d3-46b5-b3de-a2fd59e381e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:46.440228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f53ccd-13d3-46b5-b3de-a2fd59e381e6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-45f53ccd-13d3-46b5-b3de-a2fd59e381e6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:46.458734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:46.459732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:46.460661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:46.460661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:46.460661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:46.460661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:46.460661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:46.460661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:46.460661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:46.460661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:46.460661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 08:12:46.460661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:46.461195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 486/972] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Jul 20 08:12:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:46.467629 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6d78a-2598-4cb4-85ab-8a07d4a40b17 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:46.530491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:46.604632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6d78a-2598-4cb4-85ab-8a07d4a40b17 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:46.645163 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd7e8a-cce1-4df1-870b-f739c9ebadaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:46.645882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd7e8a-cce1-4df1-870b-f739c9ebadaa None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:46.646196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd7e8a-cce1-4df1-870b-f739c9ebadaa None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:46.646470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd7e8a-cce1-4df1-870b-f739c9ebadaa None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:46.647423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd7e8a-cce1-4df1-870b-f739c9ebadaa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:46.647662 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd7e8a-cce1-4df1-870b-f739c9ebadaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:46.649112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd7e8a-cce1-4df1-870b-f739c9ebadaa None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:46.670495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 487/973] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Jul 20 08:12:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:46.677671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f53ccd-13d3-46b5-b3de-a2fd59e381e6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:46.800193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f53ccd-13d3-46b5-b3de-a2fd59e381e6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:46.823749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:46.841277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220bb76-9bec-439b-bba9-37458a5da2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:46.841787 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220bb76-9bec-439b-bba9-37458a5da2de None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:46.842179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220bb76-9bec-439b-bba9-37458a5da2de None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:46.842656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220bb76-9bec-439b-bba9-37458a5da2de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:46.922972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220bb76-9bec-439b-bba9-37458a5da2de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:46.924854 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220bb76-9bec-439b-bba9-37458a5da2de None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:47.049375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 487/974] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:46 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:12:47.051119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd7e8a-cce1-4df1-870b-f739c9ebadaa None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:47.172092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd7e8a-cce1-4df1-870b-f739c9ebadaa None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:47.182003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:47.217410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf11a27-a180-4399-883b-e008ec4938e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:47.218349 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf11a27-a180-4399-883b-e008ec4938e1 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:47.218532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf11a27-a180-4399-883b-e008ec4938e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:47.218819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf11a27-a180-4399-883b-e008ec4938e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:47.233556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf11a27-a180-4399-883b-e008ec4938e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:47.236399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf11a27-a180-4399-883b-e008ec4938e1 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-cbf11a27-a180-4399-883b-e008ec4938e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:47.256944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:47.257867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:47.258771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:47.258771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:47.258771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:47.258771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:47.258771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 488/975] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:47.260295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eed4c-5e08-4576-8035-f3cc0e0008be None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:47.260771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eed4c-5e08-4576-8035-f3cc0e0008be None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:47.260847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63eed4c-5e08-4576-8035-f3cc0e0008be None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:47.262554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 488/976] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:47.264122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf11a27-a180-4399-883b-e008ec4938e1 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:47.269129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195996c2-d49b-4fbf-9bd8-12c435a59c1c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:47.269421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195996c2-d49b-4fbf-9bd8-12c435a59c1c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:47.269704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195996c2-d49b-4fbf-9bd8-12c435a59c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:47.287754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-195996c2-d49b-4fbf-9bd8-12c435a59c1c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:47.289060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-195996c2-d49b-4fbf-9bd8-12c435a59c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:47.305021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:47.362758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 489/977] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:47.364100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320fbce-1294-4be1-8bc0-ef4edc9c6b92 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:47.364392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320fbce-1294-4be1-8bc0-ef4edc9c6b92 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:47.364672 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320fbce-1294-4be1-8bc0-ef4edc9c6b92 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:47.365961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 490/978] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:47.371593 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01a818-a941-4da1-94c0-793dfd50cf51 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:47.371909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01a818-a941-4da1-94c0-793dfd50cf51 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:47.372192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01a818-a941-4da1-94c0-793dfd50cf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:47.378325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf11a27-a180-4399-883b-e008ec4938e1 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:47.391048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5a01a818-a941-4da1-94c0-793dfd50cf51 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:47.392268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5a01a818-a941-4da1-94c0-793dfd50cf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:47.415627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd16a7-9dea-43f3-89c6-b22a5d7aacf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:47.416295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd16a7-9dea-43f3-89c6-b22a5d7aacf8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:47.416585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd16a7-9dea-43f3-89c6-b22a5d7aacf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:47.416852 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd16a7-9dea-43f3-89c6-b22a5d7aacf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:47.417587 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd16a7-9dea-43f3-89c6-b22a5d7aacf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:47.417891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd16a7-9dea-43f3-89c6-b22a5d7aacf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:47.419298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd16a7-9dea-43f3-89c6-b22a5d7aacf8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:47.433340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 489/979] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:47.433753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd16a7-9dea-43f3-89c6-b22a5d7aacf8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:47.456293 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:47.465405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 491/980] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:47.466269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01a818-a941-4da1-94c0-793dfd50cf51 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:47.548786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd16a7-9dea-43f3-89c6-b22a5d7aacf8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:47.555922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:47.575931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01a818-a941-4da1-94c0-793dfd50cf51 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:47.583439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af775c7-49c8-458f-a2d3-56454fff4107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:47.584248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af775c7-49c8-458f-a2d3-56454fff4107 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:47.584533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af775c7-49c8-458f-a2d3-56454fff4107 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:47.584803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af775c7-49c8-458f-a2d3-56454fff4107 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:47.585458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af775c7-49c8-458f-a2d3-56454fff4107 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:47.586876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af775c7-49c8-458f-a2d3-56454fff4107 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-2af775c7-49c8-458f-a2d3-56454fff4107 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:47.599768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:47.600988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:47.601895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 490/981] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Jul 20 08:12:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:47.603128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af775c7-49c8-458f-a2d3-56454fff4107 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:47.614820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc925bc6-219d-4116-8756-40a5e5d90aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:47.615619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc925bc6-219d-4116-8756-40a5e5d90aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:47.615906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc925bc6-219d-4116-8756-40a5e5d90aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:47.616183 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc925bc6-219d-4116-8756-40a5e5d90aa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:47.623361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc925bc6-219d-4116-8756-40a5e5d90aa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:47.624821 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc925bc6-219d-4116-8756-40a5e5d90aa2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-fc925bc6-219d-4116-8756-40a5e5d90aa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:47.638926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:47.639866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:47.640738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:47.640738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:47.640738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:47.640738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:47.640738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:47.640738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:47.640738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:47.640738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:47.640738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 492/982] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:47.641529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc925bc6-219d-4116-8756-40a5e5d90aa2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:47.693095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:47.712451 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af775c7-49c8-458f-a2d3-56454fff4107 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:47.747291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af4711-820e-4bea-8846-da9224818366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:47.747958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af4711-820e-4bea-8846-da9224818366 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:47.748247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af4711-820e-4bea-8846-da9224818366 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:47.748530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af4711-820e-4bea-8846-da9224818366 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:47.749259 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af4711-820e-4bea-8846-da9224818366 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:47.750940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af4711-820e-4bea-8846-da9224818366 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:47.752641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc925bc6-219d-4116-8756-40a5e5d90aa2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:47.764679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 491/983] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:12:47 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:47.765520 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af4711-820e-4bea-8846-da9224818366 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:47.810862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b198778-81b1-4de5-a648-39bfa84ef469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:47.811598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b198778-81b1-4de5-a648-39bfa84ef469 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:47.811948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b198778-81b1-4de5-a648-39bfa84ef469 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:47.812253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b198778-81b1-4de5-a648-39bfa84ef469 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:47.813039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b198778-81b1-4de5-a648-39bfa84ef469 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:47.813577 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b198778-81b1-4de5-a648-39bfa84ef469 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:47.815309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b198778-81b1-4de5-a648-39bfa84ef469 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:47.817655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:47.826332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 493/984] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Jul 20 08:12:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:47.827351 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b198778-81b1-4de5-a648-39bfa84ef469 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:47.901948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af4711-820e-4bea-8846-da9224818366 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:47.956826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4232e83-f764-4135-907f-b2fcf118ebf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:47.957619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232e83-f764-4135-907f-b2fcf118ebf8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:47.958010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232e83-f764-4135-907f-b2fcf118ebf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:47.958391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4232e83-f764-4135-907f-b2fcf118ebf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:47.959242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4232e83-f764-4135-907f-b2fcf118ebf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:47.959572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4232e83-f764-4135-907f-b2fcf118ebf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:47.961028 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4232e83-f764-4135-907f-b2fcf118ebf8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:47.962421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b198778-81b1-4de5-a648-39bfa84ef469 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:47.973790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 492/985] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:47.980088 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4232e83-f764-4135-907f-b2fcf118ebf8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:47.983321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:48.010246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e77b85-6835-4a38-8952-941d6f6935df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:48.010952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e77b85-6835-4a38-8952-941d6f6935df None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:48.011239 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e77b85-6835-4a38-8952-941d6f6935df None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:48.011512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e77b85-6835-4a38-8952-941d6f6935df None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:48.012255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e77b85-6835-4a38-8952-941d6f6935df None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:48.012555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e77b85-6835-4a38-8952-941d6f6935df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:48.013945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e77b85-6835-4a38-8952-941d6f6935df None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:48.021668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 494/986] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Jul 20 08:12:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:48.027560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e77b85-6835-4a38-8952-941d6f6935df None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:48.098076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:48.101731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4232e83-f764-4135-907f-b2fcf118ebf8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:48.140130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7de5a-4130-4539-a60a-84793a429877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:48.141122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7de5a-4130-4539-a60a-84793a429877 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:48.141326 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7de5a-4130-4539-a60a-84793a429877 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:48.141550 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7de5a-4130-4539-a60a-84793a429877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:48.142629 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7de5a-4130-4539-a60a-84793a429877 None admin] RBAC: Adding query filter params (role.id=6747431cfb4944ceadd7cb581c7a23f8, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=5284fc12641540c3a42f291138b7c069) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:48.142732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e77b85-6835-4a38-8952-941d6f6935df None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:48.142952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7de5a-4130-4539-a60a-84793a429877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:48.144590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7de5a-4130-4539-a60a-84793a429877 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:48.151527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 493/987] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:47 2022] GET /identity/v3/role_assignments?user.id=5284fc12641540c3a42f291138b7c069&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=6747431cfb4944ceadd7cb581c7a23f8 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:48.183140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea201943-478d-4956-8991-4cba7ed1e790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:48.183822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea201943-478d-4956-8991-4cba7ed1e790 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:48.184104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea201943-478d-4956-8991-4cba7ed1e790 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:48.184377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea201943-478d-4956-8991-4cba7ed1e790 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:48.185200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea201943-478d-4956-8991-4cba7ed1e790 None admin] RBAC: Adding query filter params (interface=admin, service_id=ae267e0663b54eb2871df96b3e06877a, region_id=RegionOne) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:48.185518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea201943-478d-4956-8991-4cba7ed1e790 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:48.187117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea201943-478d-4956-8991-4cba7ed1e790 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:48.198675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 495/988] 10.222.0.64 () {56 vars in 1249 bytes} [Wed Jul 20 08:12:48 2022] GET /identity/v3/endpoints?service_id=ae267e0663b54eb2871df96b3e06877a&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:48.200566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7de5a-4130-4539-a60a-84793a429877 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:48.284190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:48.327101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7de5a-4130-4539-a60a-84793a429877 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:48.367329 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15322f-3577-4614-96e2-d9e6189c422a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:48.367776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15322f-3577-4614-96e2-d9e6189c422a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:48.367903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15322f-3577-4614-96e2-d9e6189c422a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:48.368150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15322f-3577-4614-96e2-d9e6189c422a None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:48.368842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15322f-3577-4614-96e2-d9e6189c422a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:48.370533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15322f-3577-4614-96e2-d9e6189c422a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:48.377186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 494/989] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Jul 20 08:12:48 2022] GET /identity/v3/services => generated 1727 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:12:49.890653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe73e1-7d74-448c-9e06-f578374cb525 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:49.890653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe73e1-7d74-448c-9e06-f578374cb525 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:49.890653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe73e1-7d74-448c-9e06-f578374cb525 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:49.892934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 496/990] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:49.898442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79417abc-bbe3-44d6-9b1f-5ef1a5292d66 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:49.898972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79417abc-bbe3-44d6-9b1f-5ef1a5292d66 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:49.899057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79417abc-bbe3-44d6-9b1f-5ef1a5292d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:49.920688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-79417abc-bbe3-44d6-9b1f-5ef1a5292d66 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:49.923390 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-79417abc-bbe3-44d6-9b1f-5ef1a5292d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:50.039224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 495/991] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:50.061749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe73e1-7d74-448c-9e06-f578374cb525 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:50.180049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe73e1-7d74-448c-9e06-f578374cb525 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:50.211498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:50.222806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd31c69-5f6e-4823-b690-8eb2762dba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:50.223807 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd31c69-5f6e-4823-b690-8eb2762dba12 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:50.223988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd31c69-5f6e-4823-b690-8eb2762dba12 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:50.224250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd31c69-5f6e-4823-b690-8eb2762dba12 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:50.224834 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd31c69-5f6e-4823-b690-8eb2762dba12 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:50.226232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd31c69-5f6e-4823-b690-8eb2762dba12 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-9bd31c69-5f6e-4823-b690-8eb2762dba12 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:50.239536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:50.241012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:50.241926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 497/992] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:50.246811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79417abc-bbe3-44d6-9b1f-5ef1a5292d66 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:50.321631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c76d3-803d-410e-b061-daec059487a2 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:50.321631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c76d3-803d-410e-b061-daec059487a2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:50.321932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c76d3-803d-410e-b061-daec059487a2 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:50.332301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:50.334425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 498/993] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:50 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:12:50.336032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f7eb-9745-4068-8e15-0cb077675feb None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:50.337310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f7eb-9745-4068-8e15-0cb077675feb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:50.337310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f7eb-9745-4068-8e15-0cb077675feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:50.357824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-01f4f7eb-9745-4068-8e15-0cb077675feb None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:50.359165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-01f4f7eb-9745-4068-8e15-0cb077675feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:50.387118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79417abc-bbe3-44d6-9b1f-5ef1a5292d66 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:50.437122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0fbc4-a36e-4f00-92cd-c226dd017fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:50.438015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0fbc4-a36e-4f00-92cd-c226dd017fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:50.438405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0fbc4-a36e-4f00-92cd-c226dd017fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:50.439249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0fbc4-a36e-4f00-92cd-c226dd017fb5 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:50.440263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0fbc4-a36e-4f00-92cd-c226dd017fb5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:50.440577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0fbc4-a36e-4f00-92cd-c226dd017fb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:50.441982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0fbc4-a36e-4f00-92cd-c226dd017fb5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:50.443611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 499/994] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:50.452375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 496/995] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:50.453463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4f7eb-9745-4068-8e15-0cb077675feb None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:50.467564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0fbc4-a36e-4f00-92cd-c226dd017fb5 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:50.562047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4f7eb-9745-4068-8e15-0cb077675feb None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:50.580031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0fbc4-a36e-4f00-92cd-c226dd017fb5 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:50.587072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:50.600217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b952a4-e37b-41f7-9e4f-8b36bef1025e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:50.600891 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b952a4-e37b-41f7-9e4f-8b36bef1025e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:50.601183 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b952a4-e37b-41f7-9e4f-8b36bef1025e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:50.601460 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b952a4-e37b-41f7-9e4f-8b36bef1025e None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:50.602179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b952a4-e37b-41f7-9e4f-8b36bef1025e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:50.602490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b952a4-e37b-41f7-9e4f-8b36bef1025e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:50.603923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b952a4-e37b-41f7-9e4f-8b36bef1025e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:50.606532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:50.610328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 500/996] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:50.612404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca92739-b7f3-4f6a-b7da-f61524601883 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:50.612897 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca92739-b7f3-4f6a-b7da-f61524601883 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:50.613176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca92739-b7f3-4f6a-b7da-f61524601883 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:50.618544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 501/997] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:50.619426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca92739-b7f3-4f6a-b7da-f61524601883 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:50.622209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff54f1-b8cf-4a2a-bcb1-672c8e7a455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:50.623053 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff54f1-b8cf-4a2a-bcb1-672c8e7a455f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:50.623340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff54f1-b8cf-4a2a-bcb1-672c8e7a455f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:50.623620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff54f1-b8cf-4a2a-bcb1-672c8e7a455f None admin] PATH_INFO: `/v3/services/image` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:50.624286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff54f1-b8cf-4a2a-bcb1-672c8e7a455f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:50.626032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff54f1-b8cf-4a2a-bcb1-672c8e7a455f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-2fff54f1-b8cf-4a2a-bcb1-672c8e7a455f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:50.638289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:50.639453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:50.640439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 497/998] 10.222.0.64 () {56 vars in 1105 bytes} [Wed Jul 20 08:12:50 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:50.643324 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a6883-991c-44c4-93c0-0870b35538bf None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:50.643831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a6883-991c-44c4-93c0-0870b35538bf None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:50.644127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a6883-991c-44c4-93c0-0870b35538bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:50.662017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-207a6883-991c-44c4-93c0-0870b35538bf None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:50.663270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-207a6883-991c-44c4-93c0-0870b35538bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:50.727549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca92739-b7f3-4f6a-b7da-f61524601883 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:50.734671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 498/999] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:50.735919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a6883-991c-44c4-93c0-0870b35538bf None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:50.774951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be975f8b-c110-46d0-a3e2-90d777831973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:50.775533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be975f8b-c110-46d0-a3e2-90d777831973 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:50.775818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be975f8b-c110-46d0-a3e2-90d777831973 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:50.776090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be975f8b-c110-46d0-a3e2-90d777831973 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:50.776883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be975f8b-c110-46d0-a3e2-90d777831973 None admin] RBAC: Adding query filter params (interface=public, service_id=ae360b87f3934d3a9977fd6eff37bc0b, region_id=RegionOne) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:50.777193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be975f8b-c110-46d0-a3e2-90d777831973 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:50.778560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be975f8b-c110-46d0-a3e2-90d777831973 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:50.789012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 502/1000] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Jul 20 08:12:50 2022] GET /identity/v3/endpoints?service_id=ae360b87f3934d3a9977fd6eff37bc0b&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:50.789012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be975f8b-c110-46d0-a3e2-90d777831973 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:50.838223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:50.844186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a6883-991c-44c4-93c0-0870b35538bf None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:50.862364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:50.881230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bad511-827e-49c3-b758-60b84e18384c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:50.881903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bad511-827e-49c3-b758-60b84e18384c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:50.882188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bad511-827e-49c3-b758-60b84e18384c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:50.882477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bad511-827e-49c3-b758-60b84e18384c None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:50.883229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bad511-827e-49c3-b758-60b84e18384c None admin] RBAC: Adding query filter params (name=image) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:50.883540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bad511-827e-49c3-b758-60b84e18384c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:50.884924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bad511-827e-49c3-b758-60b84e18384c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:50.889845 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 499/1001] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:50.890678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bad511-827e-49c3-b758-60b84e18384c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:50.890893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be975f8b-c110-46d0-a3e2-90d777831973 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:50.932921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbdc56-3be6-444a-bae4-a7fedd7f9f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:50.933734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbdc56-3be6-444a-bae4-a7fedd7f9f34 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:50.934011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbdc56-3be6-444a-bae4-a7fedd7f9f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:50.934280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbdc56-3be6-444a-bae4-a7fedd7f9f34 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:50.934963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbdc56-3be6-444a-bae4-a7fedd7f9f34 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:50.936340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbdc56-3be6-444a-bae4-a7fedd7f9f34 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e0fbdc56-3be6-444a-bae4-a7fedd7f9f34 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:50.949195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:50.950255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:50.951573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 503/1002] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Jul 20 08:12:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:50.952391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbdc56-3be6-444a-bae4-a7fedd7f9f34 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:51.007425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bad511-827e-49c3-b758-60b84e18384c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:51.044555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739bee0-17a6-4caa-b96d-75092c5f5585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:51.045220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739bee0-17a6-4caa-b96d-75092c5f5585 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:51.045507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739bee0-17a6-4caa-b96d-75092c5f5585 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:51.045788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739bee0-17a6-4caa-b96d-75092c5f5585 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:51.046472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739bee0-17a6-4caa-b96d-75092c5f5585 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:51.048081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739bee0-17a6-4caa-b96d-75092c5f5585 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:51.054591 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 500/1003] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Jul 20 08:12:50 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:12:51.054924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbdc56-3be6-444a-bae4-a7fedd7f9f34 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:51.055397 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739bee0-17a6-4caa-b96d-75092c5f5585 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:51.072159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:51.098157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908f789-6b37-4ddd-beb7-5df807b8d31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:51.098867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908f789-6b37-4ddd-beb7-5df807b8d31a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:51.099153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908f789-6b37-4ddd-beb7-5df807b8d31a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:51.099431 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e908f789-6b37-4ddd-beb7-5df807b8d31a None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:51.100155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908f789-6b37-4ddd-beb7-5df807b8d31a None admin] RBAC: Adding query filter params (type=image) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:51.100462 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908f789-6b37-4ddd-beb7-5df807b8d31a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:51.101890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e908f789-6b37-4ddd-beb7-5df807b8d31a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:51.108755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 504/1004] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:51.124402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:51.194701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739bee0-17a6-4caa-b96d-75092c5f5585 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:51.236472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6d822-9515-44be-9ab8-ebcf34e434b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:51.237175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6d822-9515-44be-9ab8-ebcf34e434b7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:51.237642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6d822-9515-44be-9ab8-ebcf34e434b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:51.237930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6d822-9515-44be-9ab8-ebcf34e434b7 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:51.238705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6d822-9515-44be-9ab8-ebcf34e434b7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:51.239021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6d822-9515-44be-9ab8-ebcf34e434b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:51.241105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6d822-9515-44be-9ab8-ebcf34e434b7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:51.247108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 501/1005] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Jul 20 08:12:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:51.252936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908f789-6b37-4ddd-beb7-5df807b8d31a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:51.368722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e908f789-6b37-4ddd-beb7-5df807b8d31a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:51.407203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62355e9-4eda-4405-bccb-d93a25700f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:51.407938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62355e9-4eda-4405-bccb-d93a25700f1e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:51.408223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62355e9-4eda-4405-bccb-d93a25700f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:51.408482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62355e9-4eda-4405-bccb-d93a25700f1e None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:51.409202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62355e9-4eda-4405-bccb-d93a25700f1e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:51.409507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62355e9-4eda-4405-bccb-d93a25700f1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:51.410926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62355e9-4eda-4405-bccb-d93a25700f1e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:51.416585 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 505/1006] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Jul 20 08:12:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:51.422160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6d822-9515-44be-9ab8-ebcf34e434b7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:51.528203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6d822-9515-44be-9ab8-ebcf34e434b7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:51.540526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:51.566166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3868924-6497-495b-8af6-a5696a03de57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:51.566852 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3868924-6497-495b-8af6-a5696a03de57 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:51.567190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3868924-6497-495b-8af6-a5696a03de57 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:51.567416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3868924-6497-495b-8af6-a5696a03de57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:51.568167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3868924-6497-495b-8af6-a5696a03de57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:51.569530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3868924-6497-495b-8af6-a5696a03de57 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:51.594788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 502/1007] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Jul 20 08:12:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:53.185019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6ad7-4616-41f0-b1c9-7041d011c288 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:53.185655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6ad7-4616-41f0-b1c9-7041d011c288 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:53.185655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5b6ad7-4616-41f0-b1c9-7041d011c288 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:53.186998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 506/1008] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:53.193103 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40ac24-1ef1-421b-b106-0681a4b5c6f5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:53.193609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40ac24-1ef1-421b-b106-0681a4b5c6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:53.193760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f40ac24-1ef1-421b-b106-0681a4b5c6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:53.211662 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6f40ac24-1ef1-421b-b106-0681a4b5c6f5 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:53.212875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6f40ac24-1ef1-421b-b106-0681a4b5c6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:53.223599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd259-ed4a-465b-9210-b70f2ce5b31d None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:53.223944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd259-ed4a-465b-9210-b70f2ce5b31d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:53.224151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfd259-ed4a-465b-9210-b70f2ce5b31d None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:53.225542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 507/1009] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:53.230929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b87ce-c52d-40d8-8dd2-4c43e6d336dc None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:53.231299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b87ce-c52d-40d8-8dd2-4c43e6d336dc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:53.231583 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b87ce-c52d-40d8-8dd2-4c43e6d336dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:53.249847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-084b87ce-c52d-40d8-8dd2-4c43e6d336dc None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:53.251979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-084b87ce-c52d-40d8-8dd2-4c43e6d336dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:53.294619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:53.302449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 503/1010] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:53 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:53.325900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40ac24-1ef1-421b-b106-0681a4b5c6f5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:53.334545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 508/1011] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:53 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:53.357713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b87ce-c52d-40d8-8dd2-4c43e6d336dc None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:53.437327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:53.447018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f40ac24-1ef1-421b-b106-0681a4b5c6f5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:53.475809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b87ce-c52d-40d8-8dd2-4c43e6d336dc None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:53.491945 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee488bd0-a819-4b2b-b5f2-ea35f9a881da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:53.493147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488bd0-a819-4b2b-b5f2-ea35f9a881da None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:53.493439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488bd0-a819-4b2b-b5f2-ea35f9a881da None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:53.493722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee488bd0-a819-4b2b-b5f2-ea35f9a881da None admin] PATH_INFO: `/v3/services/compute` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:53.494552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee488bd0-a819-4b2b-b5f2-ea35f9a881da None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:53.495975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee488bd0-a819-4b2b-b5f2-ea35f9a881da None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ee488bd0-a819-4b2b-b5f2-ea35f9a881da None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:53.508962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:53.511537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:53.513064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 504/1012] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:53.516237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee488bd0-a819-4b2b-b5f2-ea35f9a881da None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:53.521705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb76bdd-0e23-4d23-ab3e-f442350b210a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:53.522388 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb76bdd-0e23-4d23-ab3e-f442350b210a None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:53.522739 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb76bdd-0e23-4d23-ab3e-f442350b210a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:53.523160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb76bdd-0e23-4d23-ab3e-f442350b210a None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:53.523915 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb76bdd-0e23-4d23-ab3e-f442350b210a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:53.525316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb76bdd-0e23-4d23-ab3e-f442350b210a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:53.542008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 509/1013] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:12:53 2022] POST /identity/v3/services => generated 242 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:12:53.694608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee488bd0-a819-4b2b-b5f2-ea35f9a881da None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:53.717517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f91c92-6427-4f0e-b059-93b38afc1747 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:53.717809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f91c92-6427-4f0e-b059-93b38afc1747 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:53.718092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f91c92-6427-4f0e-b059-93b38afc1747 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:53.720202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 510/1014] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:12:53.726086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6561fd-6a55-4d48-b1c3-d37b2f14e174 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:53.726370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6561fd-6a55-4d48-b1c3-d37b2f14e174 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:53.726680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6561fd-6a55-4d48-b1c3-d37b2f14e174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:53.748917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-da6561fd-6a55-4d48-b1c3-d37b2f14e174 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:53.750315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-da6561fd-6a55-4d48-b1c3-d37b2f14e174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:53.751513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c88a44-7dd9-44c7-a1fe-25dd8b5171e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:53.752205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c88a44-7dd9-44c7-a1fe-25dd8b5171e4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:53.752499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c88a44-7dd9-44c7-a1fe-25dd8b5171e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:53.752783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c88a44-7dd9-44c7-a1fe-25dd8b5171e4 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:53.753518 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c88a44-7dd9-44c7-a1fe-25dd8b5171e4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:53.753836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c88a44-7dd9-44c7-a1fe-25dd8b5171e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:53.755296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c88a44-7dd9-44c7-a1fe-25dd8b5171e4 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:53.760877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 505/1015] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:53.767656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c88a44-7dd9-44c7-a1fe-25dd8b5171e4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:53.794947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:53.796790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:53.857086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 511/1016] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:53 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:53.875048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6561fd-6a55-4d48-b1c3-d37b2f14e174 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:53.899119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c88a44-7dd9-44c7-a1fe-25dd8b5171e4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:53.946522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3b1e8-a6c7-4669-af41-bd9a52598f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:53.947593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3b1e8-a6c7-4669-af41-bd9a52598f1c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:53.948051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3b1e8-a6c7-4669-af41-bd9a52598f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:53.948477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3b1e8-a6c7-4669-af41-bd9a52598f1c None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:53.949348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3b1e8-a6c7-4669-af41-bd9a52598f1c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:53.949823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3b1e8-a6c7-4669-af41-bd9a52598f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:53.951607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3b1e8-a6c7-4669-af41-bd9a52598f1c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:53.958090 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 506/1017] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Jul 20 08:12:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:53.963500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3b1e8-a6c7-4669-af41-bd9a52598f1c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:54.007049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6561fd-6a55-4d48-b1c3-d37b2f14e174 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:54.049874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24464b3-7736-4af3-8101-acd66a110e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:54.051115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24464b3-7736-4af3-8101-acd66a110e2c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:54.051495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24464b3-7736-4af3-8101-acd66a110e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:54.052772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24464b3-7736-4af3-8101-acd66a110e2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:54.054184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24464b3-7736-4af3-8101-acd66a110e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:54.055795 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24464b3-7736-4af3-8101-acd66a110e2c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:54.058631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 512/1018] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:54.065816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24464b3-7736-4af3-8101-acd66a110e2c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:54.078732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:54.098078 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3b1e8-a6c7-4669-af41-bd9a52598f1c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:54.099330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:54.138892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2dce8-cae7-4436-a332-67dfb869377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:54.139561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2dce8-cae7-4436-a332-67dfb869377f None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:54.139849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2dce8-cae7-4436-a332-67dfb869377f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:54.140133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2dce8-cae7-4436-a332-67dfb869377f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:54.140906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2dce8-cae7-4436-a332-67dfb869377f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:54.142302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2dce8-cae7-4436-a332-67dfb869377f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:54.186617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 507/1019] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Jul 20 08:12:53 2022] POST /identity/v3/endpoints => generated 334 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:12:54.199931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24464b3-7736-4af3-8101-acd66a110e2c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:54.249011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3358e-578a-4197-b83c-d175c71b87d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:54.249691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3358e-578a-4197-b83c-d175c71b87d6 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:54.249974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3358e-578a-4197-b83c-d175c71b87d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:54.250417 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3358e-578a-4197-b83c-d175c71b87d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:54.251257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3358e-578a-4197-b83c-d175c71b87d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:54.252754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3358e-578a-4197-b83c-d175c71b87d6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:54.285170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 513/1020] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:12:54 2022] POST /identity/v3/projects => generated 311 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:55.805839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d47506-4909-4c7a-81b5-ab91dea73e97 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:55.806386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d47506-4909-4c7a-81b5-ab91dea73e97 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:55.806386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d47506-4909-4c7a-81b5-ab91dea73e97 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:55.808544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 508/1021] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:55.815007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec586ec-2087-4dd3-918a-452f33387f04 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:55.815657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec586ec-2087-4dd3-918a-452f33387f04 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:55.815657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec586ec-2087-4dd3-918a-452f33387f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:55.837930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2ec586ec-2087-4dd3-918a-452f33387f04 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:55.840873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2ec586ec-2087-4dd3-918a-452f33387f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:55.955337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 514/1022] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:12:55.983667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d47506-4909-4c7a-81b5-ab91dea73e97 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:56.143691 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d47506-4909-4c7a-81b5-ab91dea73e97 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:56.202307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0686a-75d8-4a8f-b708-e95e01e67d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:56.203657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0686a-75d8-4a8f-b708-e95e01e67d2c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:56.204153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0686a-75d8-4a8f-b708-e95e01e67d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:56.204616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0686a-75d8-4a8f-b708-e95e01e67d2c None admin] PATH_INFO: `/v3/services/image` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:56.205480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0686a-75d8-4a8f-b708-e95e01e67d2c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:56.207129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0686a-75d8-4a8f-b708-e95e01e67d2c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-80f0686a-75d8-4a8f-b708-e95e01e67d2c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:56.220650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:56.221773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 08:12:56.222751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:56.224430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 509/1023] 10.222.0.64 () {56 vars in 1105 bytes} [Wed Jul 20 08:12:55 2022] GET /identity/v3/services/image => generated 86 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:56.232930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec586ec-2087-4dd3-918a-452f33387f04 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:56.247478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:56.378112 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec586ec-2087-4dd3-918a-452f33387f04 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:56.424471 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ab0ca-2882-4505-8fb4-db81cfbf8039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:56.425063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ab0ca-2882-4505-8fb4-db81cfbf8039 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:56.425341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ab0ca-2882-4505-8fb4-db81cfbf8039 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:56.425613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ab0ca-2882-4505-8fb4-db81cfbf8039 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:56.426341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ab0ca-2882-4505-8fb4-db81cfbf8039 None admin] RBAC: Adding query filter params (name=image) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:56.426678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ab0ca-2882-4505-8fb4-db81cfbf8039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:56.428088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ab0ca-2882-4505-8fb4-db81cfbf8039 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:56.435964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 515/1024] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:56.443312 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0686a-75d8-4a8f-b708-e95e01e67d2c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:56.448422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:56.492518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6e0e-6f0d-407d-b0f9-a5db3a769489 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:56.492887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6e0e-6f0d-407d-b0f9-a5db3a769489 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:56.493210 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af6e0e-6f0d-407d-b0f9-a5db3a769489 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:56.496487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 516/1025] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:56.503367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298821-50d1-48d7-a2f3-04ad0d934664 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:56.503367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298821-50d1-48d7-a2f3-04ad0d934664 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:56.503367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b298821-50d1-48d7-a2f3-04ad0d934664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:56.527079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3b298821-50d1-48d7-a2f3-04ad0d934664 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:56.529163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3b298821-50d1-48d7-a2f3-04ad0d934664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:56.589483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0686a-75d8-4a8f-b708-e95e01e67d2c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:56.615593 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 517/1026] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:56.616760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9f9f3-5464-4349-a8c3-b418cb33c5c2 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:56.617058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9f9f3-5464-4349-a8c3-b418cb33c5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:56.617377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9f9f3-5464-4349-a8c3-b418cb33c5c2 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:56.623198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 518/1027] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:12:56.625048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924c25b-b59b-4470-ac87-10278246537e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:56.625399 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924c25b-b59b-4470-ac87-10278246537e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:56.625762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924c25b-b59b-4470-ac87-10278246537e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:56.628443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47977db8-24b3-45d3-b22f-0008b53eb355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:56.629120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47977db8-24b3-45d3-b22f-0008b53eb355 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:56.629406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47977db8-24b3-45d3-b22f-0008b53eb355 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:56.629687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47977db8-24b3-45d3-b22f-0008b53eb355 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:56.630409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47977db8-24b3-45d3-b22f-0008b53eb355 None admin] RBAC: Adding query filter params (type=image) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:56.630759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47977db8-24b3-45d3-b22f-0008b53eb355 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:56.632161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47977db8-24b3-45d3-b22f-0008b53eb355 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:56.637622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 510/1028] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:56.639619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:56.643592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47977db8-24b3-45d3-b22f-0008b53eb355 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:56.646717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9924c25b-b59b-4470-ac87-10278246537e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:56.647963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9924c25b-b59b-4470-ac87-10278246537e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:56.731384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 519/1029] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:56.732001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:56.732153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924c25b-b59b-4470-ac87-10278246537e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:56.775942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47977db8-24b3-45d3-b22f-0008b53eb355 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:56.812444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d152592-764a-41c3-b57c-2d81ea253189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:56.813370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152592-764a-41c3-b57c-2d81ea253189 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:56.813657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152592-764a-41c3-b57c-2d81ea253189 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:56.813940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d152592-764a-41c3-b57c-2d81ea253189 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:56.815201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d152592-764a-41c3-b57c-2d81ea253189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:56.816651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d152592-764a-41c3-b57c-2d81ea253189 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:56.818336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 511/1030] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:56.819122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d152592-764a-41c3-b57c-2d81ea253189 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:56.855023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924c25b-b59b-4470-ac87-10278246537e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:56.899644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:56.905961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512face0-9a14-4416-a51d-227eb3eea889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:56.906902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512face0-9a14-4416-a51d-227eb3eea889 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:56.907419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512face0-9a14-4416-a51d-227eb3eea889 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:56.907718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512face0-9a14-4416-a51d-227eb3eea889 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:56.908544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512face0-9a14-4416-a51d-227eb3eea889 None admin] RBAC: Adding query filter params (interface=public, service_id=a5710e896d824e14a722c9fd6b4325c6, region_id=RegionOne) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:56.908854 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512face0-9a14-4416-a51d-227eb3eea889 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:56.910397 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512face0-9a14-4416-a51d-227eb3eea889 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:56.918563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 520/1031] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Jul 20 08:12:56 2022] GET /identity/v3/endpoints?service_id=a5710e896d824e14a722c9fd6b4325c6&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:56.919459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512face0-9a14-4416-a51d-227eb3eea889 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:56.949606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d152592-764a-41c3-b57c-2d81ea253189 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:56.990490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f1e12-a483-49c3-a5bc-341600844a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:56.991266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f1e12-a483-49c3-a5bc-341600844a54 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:56.991555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f1e12-a483-49c3-a5bc-341600844a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:56.991831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f1e12-a483-49c3-a5bc-341600844a54 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:56.995032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:57.008086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f1e12-a483-49c3-a5bc-341600844a54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d35f1e12-a483-49c3-a5bc-341600844a54 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:57.021327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:57.022624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 512/1032] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Jul 20 08:12:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:57.024667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f1e12-a483-49c3-a5bc-341600844a54 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:57.052268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512face0-9a14-4416-a51d-227eb3eea889 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:57.093046 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f3e3-fea6-4d02-9c6c-19fbce84e9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:57.093710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f3e3-fea6-4d02-9c6c-19fbce84e9d7 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:57.094605 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f3e3-fea6-4d02-9c6c-19fbce84e9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:57.094868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659f3e3-fea6-4d02-9c6c-19fbce84e9d7 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:57.095584 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659f3e3-fea6-4d02-9c6c-19fbce84e9d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:57.097034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659f3e3-fea6-4d02-9c6c-19fbce84e9d7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:57.118567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 521/1033] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:12:56 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:12:57.119413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f3e3-fea6-4d02-9c6c-19fbce84e9d7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:57.158125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f1e12-a483-49c3-a5bc-341600844a54 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:57.192036 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:57.203271 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a6457-45ab-4263-a361-1a7a22dbe88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:57.203977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a6457-45ab-4263-a361-1a7a22dbe88d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:57.204276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a6457-45ab-4263-a361-1a7a22dbe88d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:57.204649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a6457-45ab-4263-a361-1a7a22dbe88d None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:57.205364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a6457-45ab-4263-a361-1a7a22dbe88d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:57.206803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a6457-45ab-4263-a361-1a7a22dbe88d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:57.215076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 513/1034] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Jul 20 08:12:57 2022] GET /identity/v3/services => generated 1957 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:12:57.256580 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659f3e3-fea6-4d02-9c6c-19fbce84e9d7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:57.302469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a10b0-b711-46e8-b16d-f60f33ea0ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:57.303191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a10b0-b711-46e8-b16d-f60f33ea0ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:57.303479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a10b0-b711-46e8-b16d-f60f33ea0ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:57.303756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a10b0-b711-46e8-b16d-f60f33ea0ec3 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:57.304539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a10b0-b711-46e8-b16d-f60f33ea0ec3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:57.304847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a10b0-b711-46e8-b16d-f60f33ea0ec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:57.306294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a10b0-b711-46e8-b16d-f60f33ea0ec3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:57.317456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 522/1035] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Jul 20 08:12:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:57.323414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a6457-45ab-4263-a361-1a7a22dbe88d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:57.400686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:57.452691 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a6457-45ab-4263-a361-1a7a22dbe88d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:57.490601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707e01d-2e1e-4369-aca7-a4bfca6d6f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:57.491422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707e01d-2e1e-4369-aca7-a4bfca6d6f6e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:57.491709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707e01d-2e1e-4369-aca7-a4bfca6d6f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:57.491987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707e01d-2e1e-4369-aca7-a4bfca6d6f6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:57.504045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707e01d-2e1e-4369-aca7-a4bfca6d6f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:57.505513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707e01d-2e1e-4369-aca7-a4bfca6d6f6e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-2707e01d-2e1e-4369-aca7-a4bfca6d6f6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:57.524057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:57.533759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:57.535491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:57.535491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:57.535491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:57.535491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:57.535491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 514/1036] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:57.535844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a10b0-b711-46e8-b16d-f60f33ea0ec3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:57.642894 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a10b0-b711-46e8-b16d-f60f33ea0ec3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:57.658432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:57.681669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06977345-5a8e-44c5-b30b-fcd4526dac8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:57.682340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06977345-5a8e-44c5-b30b-fcd4526dac8b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:57.682668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06977345-5a8e-44c5-b30b-fcd4526dac8b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:57.682950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06977345-5a8e-44c5-b30b-fcd4526dac8b None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:57.683689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06977345-5a8e-44c5-b30b-fcd4526dac8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:57.683999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06977345-5a8e-44c5-b30b-fcd4526dac8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:57.685402 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06977345-5a8e-44c5-b30b-fcd4526dac8b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:57.705355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 523/1037] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:57.706362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707e01d-2e1e-4369-aca7-a4bfca6d6f6e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:57.816559 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707e01d-2e1e-4369-aca7-a4bfca6d6f6e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:57.830938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:57.854592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afb7a6-f70c-48b9-9b93-844802a4949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:57.855280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb7a6-f70c-48b9-9b93-844802a4949e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:57.855566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb7a6-f70c-48b9-9b93-844802a4949e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:57.855843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68afb7a6-f70c-48b9-9b93-844802a4949e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:57.866826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afb7a6-f70c-48b9-9b93-844802a4949e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:57.868218 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68afb7a6-f70c-48b9-9b93-844802a4949e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-68afb7a6-f70c-48b9-9b93-844802a4949e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:12:57.884881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:57.886097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:57.887553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:57.887553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:57.887553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:12:57.887553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:12:57.887553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:12:57.887553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:12:57.887553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:12:57.887553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:12:57.887553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:12:57.887553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:57.888441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 515/1038] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Jul 20 08:12:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:57.892400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06977345-5a8e-44c5-b30b-fcd4526dac8b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:58.014215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06977345-5a8e-44c5-b30b-fcd4526dac8b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:58.055379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba96ef-07df-407e-a93f-ccd9502d7ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:58.056074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba96ef-07df-407e-a93f-ccd9502d7ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:58.056364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba96ef-07df-407e-a93f-ccd9502d7ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:58.056638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba96ef-07df-407e-a93f-ccd9502d7ea6 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:58.057508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba96ef-07df-407e-a93f-ccd9502d7ea6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:58.057823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba96ef-07df-407e-a93f-ccd9502d7ea6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:58.059383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba96ef-07df-407e-a93f-ccd9502d7ea6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:58.082028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 524/1039] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Jul 20 08:12:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:58.087490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afb7a6-f70c-48b9-9b93-844802a4949e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:58.195168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68afb7a6-f70c-48b9-9b93-844802a4949e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:58.230276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee2efb-3e55-41ae-9e29-d858f1edaa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:58.231041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee2efb-3e55-41ae-9e29-d858f1edaa7d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:58.231333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee2efb-3e55-41ae-9e29-d858f1edaa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:58.231613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee2efb-3e55-41ae-9e29-d858f1edaa7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:58.288757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:58.318037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee2efb-3e55-41ae-9e29-d858f1edaa7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:58.319538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee2efb-3e55-41ae-9e29-d858f1edaa7d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:58.433386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 516/1040] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:12:58 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:12:58.442975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba96ef-07df-407e-a93f-ccd9502d7ea6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:58.475259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:58.551374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba96ef-07df-407e-a93f-ccd9502d7ea6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:58.587632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988bb3a-c92c-4cc6-bc41-241f9293488d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:58.588412 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988bb3a-c92c-4cc6-bc41-241f9293488d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:58.588692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988bb3a-c92c-4cc6-bc41-241f9293488d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:58.588962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988bb3a-c92c-4cc6-bc41-241f9293488d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:58.600719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988bb3a-c92c-4cc6-bc41-241f9293488d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:58.602182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f988bb3a-c92c-4cc6-bc41-241f9293488d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-f988bb3a-c92c-4cc6-bc41-241f9293488d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:12:58.625827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:58.626921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:58.632875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:12:58.632875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:12:58.632875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:12:58.632875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:58.632875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 525/1041] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:58.633172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee2efb-3e55-41ae-9e29-d858f1edaa7d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:58.708420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:58.741726 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee2efb-3e55-41ae-9e29-d858f1edaa7d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:58.780133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23fef4-27c3-449b-a4e8-99ac68a7853b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:58.780805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23fef4-27c3-449b-a4e8-99ac68a7853b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:58.781156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23fef4-27c3-449b-a4e8-99ac68a7853b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:58.781466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23fef4-27c3-449b-a4e8-99ac68a7853b None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:58.782224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23fef4-27c3-449b-a4e8-99ac68a7853b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:58.782534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23fef4-27c3-449b-a4e8-99ac68a7853b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:58.783957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb23fef4-27c3-449b-a4e8-99ac68a7853b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:58.796212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 517/1042] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:58.801797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988bb3a-c92c-4cc6-bc41-241f9293488d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:58.893484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:58.913104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f988bb3a-c92c-4cc6-bc41-241f9293488d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:58.951281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e359f64-f91f-4918-a576-4a14884a1620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:58.952073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e359f64-f91f-4918-a576-4a14884a1620 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:58.952356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e359f64-f91f-4918-a576-4a14884a1620 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:58.952621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e359f64-f91f-4918-a576-4a14884a1620 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:58.959914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e359f64-f91f-4918-a576-4a14884a1620 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:58.961300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e359f64-f91f-4918-a576-4a14884a1620 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-9e359f64-f91f-4918-a576-4a14884a1620 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:12:58.975717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:58.976715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:58.977579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:12:58.977579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:12:58.977579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:12:58.977579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:12:58.977579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:12:58.977579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:12:58.977579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:12:58.977579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:58.978950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 526/1043] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:12:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:12:58.983084 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23fef4-27c3-449b-a4e8-99ac68a7853b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:59.105419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23fef4-27c3-449b-a4e8-99ac68a7853b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:59.138235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:59.145698 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9086ec-5161-456f-946d-0ab83e4da334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:59.146373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9086ec-5161-456f-946d-0ab83e4da334 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:59.146684 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9086ec-5161-456f-946d-0ab83e4da334 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:59.146964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9086ec-5161-456f-946d-0ab83e4da334 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:59.147850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9086ec-5161-456f-946d-0ab83e4da334 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:59.148161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9086ec-5161-456f-946d-0ab83e4da334 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:59.149601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9086ec-5161-456f-946d-0ab83e4da334 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:59.164522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 518/1044] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:12:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:59.170524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e359f64-f91f-4918-a576-4a14884a1620 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:59.287987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e359f64-f91f-4918-a576-4a14884a1620 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:59.295097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af465b7b-52ed-470e-ba18-b0fbb39320ba None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:59.295275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af465b7b-52ed-470e-ba18-b0fbb39320ba None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:59.295536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af465b7b-52ed-470e-ba18-b0fbb39320ba None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:59.297771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 519/1045] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:12:59.303282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f264-41a0-484a-b5f3-e39a35424a0f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:59.303282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f264-41a0-484a-b5f3-e39a35424a0f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:59.303623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236f264-41a0-484a-b5f3-e39a35424a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:59.322922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3236f264-41a0-484a-b5f3-e39a35424a0f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:59.324265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3236f264-41a0-484a-b5f3-e39a35424a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:59.330776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42b1223-4a3d-4148-a03c-a9d9065957d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:59.331743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b1223-4a3d-4148-a03c-a9d9065957d4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:59.332208 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b1223-4a3d-4148-a03c-a9d9065957d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:59.332680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b1223-4a3d-4148-a03c-a9d9065957d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:59.333704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b1223-4a3d-4148-a03c-a9d9065957d4 None admin] RBAC: Adding query filter params (role.id=6747431cfb4944ceadd7cb581c7a23f8, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=84f0da8709624834976d0662aeb02ea6) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:59.334215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b1223-4a3d-4148-a03c-a9d9065957d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:59.336068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42b1223-4a3d-4148-a03c-a9d9065957d4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:59.350628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 527/1046] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:12:59 2022] GET /identity/v3/role_assignments?user.id=84f0da8709624834976d0662aeb02ea6&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=6747431cfb4944ceadd7cb581c7a23f8 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:12:59.395561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee676-19fe-4035-992a-6a3523bdb207 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:59.395911 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee676-19fe-4035-992a-6a3523bdb207 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:59.396136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ee676-19fe-4035-992a-6a3523bdb207 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:59.397881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 528/1047] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:12:59.404600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f0e91-08bd-4fc7-b8b8-7d9e25aafb2b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:59.404908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f0e91-08bd-4fc7-b8b8-7d9e25aafb2b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:59.405192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f0e91-08bd-4fc7-b8b8-7d9e25aafb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:59.416614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 520/1048] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:59.426104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-302f0e91-08bd-4fc7-b8b8-7d9e25aafb2b None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:12:59.427796 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-302f0e91-08bd-4fc7-b8b8-7d9e25aafb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:12:59.430373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:59.442096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f264-41a0-484a-b5f3-e39a35424a0f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:59.524632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 529/1049] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:12:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:12:59.548933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f0e91-08bd-4fc7-b8b8-7d9e25aafb2b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:59.579336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236f264-41a0-484a-b5f3-e39a35424a0f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:59.608525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:59.622073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6649c63-fb89-4894-9a8c-5dfd5153a65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:59.622916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6649c63-fb89-4894-9a8c-5dfd5153a65a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:59.623196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6649c63-fb89-4894-9a8c-5dfd5153a65a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:59.623468 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6649c63-fb89-4894-9a8c-5dfd5153a65a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:59.635020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6649c63-fb89-4894-9a8c-5dfd5153a65a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b6649c63-fb89-4894-9a8c-5dfd5153a65a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:12:59.645011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:59.646026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:12:59.646881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 521/1050] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:12:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:59.652516 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6649c63-fb89-4894-9a8c-5dfd5153a65a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:59.667860 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f0e91-08bd-4fc7-b8b8-7d9e25aafb2b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:59.712696 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a2d2e-08a3-45ff-b2e1-d5f79ba992f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:59.713539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a2d2e-08a3-45ff-b2e1-d5f79ba992f4 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:59.713860 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a2d2e-08a3-45ff-b2e1-d5f79ba992f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:59.714339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a2d2e-08a3-45ff-b2e1-d5f79ba992f4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:59.714826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a2d2e-08a3-45ff-b2e1-d5f79ba992f4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:59.716224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a2d2e-08a3-45ff-b2e1-d5f79ba992f4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-e81a2d2e-08a3-45ff-b2e1-d5f79ba992f4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:12:59.729379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:12:59.730701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:12:59.731560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 530/1051] 10.222.0.64 () {56 vars in 1105 bytes} [Wed Jul 20 08:12:59 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:12:59.737623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a2d2e-08a3-45ff-b2e1-d5f79ba992f4 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:59.740550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:59.775386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6649c63-fb89-4894-9a8c-5dfd5153a65a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:59.810803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f6264-2815-4660-9023-79219e42f1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:59.811447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f6264-2815-4660-9023-79219e42f1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:59.811729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f6264-2815-4660-9023-79219e42f1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:59.812031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f6264-2815-4660-9023-79219e42f1e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:59.813546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f6264-2815-4660-9023-79219e42f1e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:59.813964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f6264-2815-4660-9023-79219e42f1e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:59.815441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f6264-2815-4660-9023-79219e42f1e7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:59.824771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 522/1052] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:12:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:59.830669 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f6264-2815-4660-9023-79219e42f1e7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:59.854156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a2d2e-08a3-45ff-b2e1-d5f79ba992f4 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:59.887494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:12:59.900858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ea4ef-aa9e-4f53-9a57-ca44d31c3f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:12:59.901440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ea4ef-aa9e-4f53-9a57-ca44d31c3f55 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:12:59.901799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ea4ef-aa9e-4f53-9a57-ca44d31c3f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:12:59.902089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ea4ef-aa9e-4f53-9a57-ca44d31c3f55 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:12:59.902836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ea4ef-aa9e-4f53-9a57-ca44d31c3f55 None admin] RBAC: Adding query filter params (name=image) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:12:59.903155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ea4ef-aa9e-4f53-9a57-ca44d31c3f55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:12:59.904557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ea4ef-aa9e-4f53-9a57-ca44d31c3f55 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:12:59.915823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 531/1053] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:12:59.921617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ea4ef-aa9e-4f53-9a57-ca44d31c3f55 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:12:59.960335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f6264-2815-4660-9023-79219e42f1e7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:12:59.999420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:00.000984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe964a89-a2b2-4d28-83ae-521400896d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:00.002485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe964a89-a2b2-4d28-83ae-521400896d15 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:00.002485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe964a89-a2b2-4d28-83ae-521400896d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:00.002485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe964a89-a2b2-4d28-83ae-521400896d15 None admin] PATH_INFO: `/v3/users/35fb90211c9346a98851ccc1fcd4385a` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:00.017881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe964a89-a2b2-4d28-83ae-521400896d15 None admin] RBAC: Authorizing `identity:get_user(user_id=35fb90211c9346a98851ccc1fcd4385a)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:00.019488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe964a89-a2b2-4d28-83ae-521400896d15 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:00.021310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 523/1054] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:12:59 2022] GET /identity/v3/users/35fb90211c9346a98851ccc1fcd4385a => generated 285 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:00.027211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe964a89-a2b2-4d28-83ae-521400896d15 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:00.069005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ea4ef-aa9e-4f53-9a57-ca44d31c3f55 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:00.107470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a45f00-21b6-4af1-b7c5-075d68082f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:00.108141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a45f00-21b6-4af1-b7c5-075d68082f5f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:00.108425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a45f00-21b6-4af1-b7c5-075d68082f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:00.108701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a45f00-21b6-4af1-b7c5-075d68082f5f None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:00.109487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a45f00-21b6-4af1-b7c5-075d68082f5f None admin] RBAC: Adding query filter params (type=image) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:00.110484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a45f00-21b6-4af1-b7c5-075d68082f5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:00.112159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a45f00-21b6-4af1-b7c5-075d68082f5f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:00.117430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 532/1055] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:12:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:00.123177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a45f00-21b6-4af1-b7c5-075d68082f5f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:00.205625 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe964a89-a2b2-4d28-83ae-521400896d15 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:00.208902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:00.236180 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a45f00-21b6-4af1-b7c5-075d68082f5f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:00.250922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dc067-cdc8-4ccb-b772-5913fa0357ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:00.251695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dc067-cdc8-4ccb-b772-5913fa0357ac None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:00.251980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dc067-cdc8-4ccb-b772-5913fa0357ac None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:00.252250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dc067-cdc8-4ccb-b772-5913fa0357ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:00.272614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:00.280684 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dcd51-e8e6-4135-9e31-b8ece2e81724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:00.281567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dcd51-e8e6-4135-9e31-b8ece2e81724 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:00.281849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dcd51-e8e6-4135-9e31-b8ece2e81724 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:00.282160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dcd51-e8e6-4135-9e31-b8ece2e81724 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:00.282999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76dcd51-e8e6-4135-9e31-b8ece2e81724 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:00.284301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76dcd51-e8e6-4135-9e31-b8ece2e81724 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:00.305148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 533/1056] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Jul 20 08:13:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:13:00.336837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1dc067-cdc8-4ccb-b772-5913fa0357ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:00.338834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1dc067-cdc8-4ccb-b772-5913fa0357ac None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:00.462091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 524/1057] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:13:00 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 08:13:00.464097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dcd51-e8e6-4135-9e31-b8ece2e81724 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:00.587316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dcd51-e8e6-4135-9e31-b8ece2e81724 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:00.626440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753712f-e3e7-4ea6-b3f3-42d59ff61f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:00.627345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753712f-e3e7-4ea6-b3f3-42d59ff61f94 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:00.627638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753712f-e3e7-4ea6-b3f3-42d59ff61f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:00.627918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753712f-e3e7-4ea6-b3f3-42d59ff61f94 None admin] PATH_INFO: `/v3/projects/04e3033a1c75463fb773a9a8587eb50b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:00.629135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753712f-e3e7-4ea6-b3f3-42d59ff61f94 None admin] RBAC: Authorizing `identity:get_project(project_id=04e3033a1c75463fb773a9a8587eb50b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:00.630617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753712f-e3e7-4ea6-b3f3-42d59ff61f94 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:00.632544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 534/1058] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:00 2022] GET /identity/v3/projects/04e3033a1c75463fb773a9a8587eb50b => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:00.638824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dc067-cdc8-4ccb-b772-5913fa0357ac None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:00.641753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:00.754496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dc067-cdc8-4ccb-b772-5913fa0357ac None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:00.791517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fedf8-3068-4aa9-848f-5e2358d599d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:00.792176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fedf8-3068-4aa9-848f-5e2358d599d2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:00.792459 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fedf8-3068-4aa9-848f-5e2358d599d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:00.792736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fedf8-3068-4aa9-848f-5e2358d599d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:00.793563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fedf8-3068-4aa9-848f-5e2358d599d2 None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=04e3033a1c75463fb773a9a8587eb50b, user.id=35fb90211c9346a98851ccc1fcd4385a) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:00.793877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fedf8-3068-4aa9-848f-5e2358d599d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:00.795346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fedf8-3068-4aa9-848f-5e2358d599d2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:00.810626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 525/1059] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:13:00 2022] GET /identity/v3/role_assignments?user.id=35fb90211c9346a98851ccc1fcd4385a&scope.project.id=04e3033a1c75463fb773a9a8587eb50b&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:01.462663 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485684aa-4562-479d-85e5-df9294c99415 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:01.463229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485684aa-4562-479d-85e5-df9294c99415 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:01.463229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485684aa-4562-479d-85e5-df9294c99415 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:01.464646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 535/1060] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:01.470541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c92c8f-e67e-421e-a003-758fe470b26b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:01.471052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c92c8f-e67e-421e-a003-758fe470b26b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:01.471131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c92c8f-e67e-421e-a003-758fe470b26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:01.489549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-84c92c8f-e67e-421e-a003-758fe470b26b None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:01.490837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-84c92c8f-e67e-421e-a003-758fe470b26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:01.556760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:01.566310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 526/1061] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:01.593563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485684aa-4562-479d-85e5-df9294c99415 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:01.648481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:01.723735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485684aa-4562-479d-85e5-df9294c99415 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:01.765547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d6836-b10d-4c94-9b01-cc5eeee12c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:01.767246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d6836-b10d-4c94-9b01-cc5eeee12c10 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:01.767540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d6836-b10d-4c94-9b01-cc5eeee12c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:01.767816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d6836-b10d-4c94-9b01-cc5eeee12c10 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:01.778916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d6836-b10d-4c94-9b01-cc5eeee12c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-316d6836-b10d-4c94-9b01-cc5eeee12c10 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:13:01.787686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:01.789525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:01.790877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 536/1062] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Jul 20 08:13:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:01.796049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c92c8f-e67e-421e-a003-758fe470b26b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:01.909947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c92c8f-e67e-421e-a003-758fe470b26b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:01.952515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25fb76-1d4e-450b-92b6-c31f83fd3b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:01.954112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25fb76-1d4e-450b-92b6-c31f83fd3b64 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:01.954112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25fb76-1d4e-450b-92b6-c31f83fd3b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:01.954112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25fb76-1d4e-450b-92b6-c31f83fd3b64 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:01.954925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25fb76-1d4e-450b-92b6-c31f83fd3b64 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:01.955243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25fb76-1d4e-450b-92b6-c31f83fd3b64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:01.956720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25fb76-1d4e-450b-92b6-c31f83fd3b64 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:01.971867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 527/1063] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Jul 20 08:13:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:01.972861 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d6836-b10d-4c94-9b01-cc5eeee12c10 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:02.103715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d6836-b10d-4c94-9b01-cc5eeee12c10 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:02.118127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:02.149254 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5727e23-29b7-4b41-bfc1-2469255bc716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:02.150332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5727e23-29b7-4b41-bfc1-2469255bc716 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:02.150809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5727e23-29b7-4b41-bfc1-2469255bc716 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:02.151260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5727e23-29b7-4b41-bfc1-2469255bc716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:02.164339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5727e23-29b7-4b41-bfc1-2469255bc716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:02.166124 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5727e23-29b7-4b41-bfc1-2469255bc716 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-d5727e23-29b7-4b41-bfc1-2469255bc716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:13:02.186300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:02.187365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:02.188900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:13:02.188900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:13:02.188900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:13:02.188900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:02.189495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 537/1064] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:13:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:02.195470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25fb76-1d4e-450b-92b6-c31f83fd3b64 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:02.265971 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:02.323042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25fb76-1d4e-450b-92b6-c31f83fd3b64 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:02.362889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7ae3e-335e-4d5d-ac6b-df3398031ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:02.363561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7ae3e-335e-4d5d-ac6b-df3398031ddb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:02.363848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7ae3e-335e-4d5d-ac6b-df3398031ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:02.364115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7ae3e-335e-4d5d-ac6b-df3398031ddb None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:02.364843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7ae3e-335e-4d5d-ac6b-df3398031ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:02.365149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7ae3e-335e-4d5d-ac6b-df3398031ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:02.366522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7ae3e-335e-4d5d-ac6b-df3398031ddb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:02.384761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 528/1065] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:02.386321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5727e23-29b7-4b41-bfc1-2469255bc716 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:02.464424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306d2e5-21d5-4611-9e42-279551734457 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:02.464818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306d2e5-21d5-4611-9e42-279551734457 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:02.465042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306d2e5-21d5-4611-9e42-279551734457 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:02.466564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 529/1066] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:13:02.472162 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fd606-3bb4-4ba2-bfd5-6810e050c906 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:02.472751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fd606-3bb4-4ba2-bfd5-6810e050c906 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:02.472751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020fd606-3bb4-4ba2-bfd5-6810e050c906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:02.490824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-020fd606-3bb4-4ba2-bfd5-6810e050c906 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:02.492035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-020fd606-3bb4-4ba2-bfd5-6810e050c906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:02.498327 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5727e23-29b7-4b41-bfc1-2469255bc716 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:02.539880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700aa8f7-3559-473e-bc06-f28741110e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:02.540721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700aa8f7-3559-473e-bc06-f28741110e7f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:02.540967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700aa8f7-3559-473e-bc06-f28741110e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:02.541237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700aa8f7-3559-473e-bc06-f28741110e7f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:02.551170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700aa8f7-3559-473e-bc06-f28741110e7f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:02.552782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700aa8f7-3559-473e-bc06-f28741110e7f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-700aa8f7-3559-473e-bc06-f28741110e7f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:13:02.568269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:02.569299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:02.570207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:02.570207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:02.570207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:13:02.570207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:13:02.570207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:13:02.570207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:13:02.570207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:13:02.570207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:13:02.570207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:13:02.570207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:02.570207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 538/1067] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Jul 20 08:13:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:02.575220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700aa8f7-3559-473e-bc06-f28741110e7f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:02.575774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 530/1068] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:13:02.599782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fd606-3bb4-4ba2-bfd5-6810e050c906 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:02.609355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:02.659833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:02.710636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700aa8f7-3559-473e-bc06-f28741110e7f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:02.725325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020fd606-3bb4-4ba2-bfd5-6810e050c906 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:02.746077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed7cde-537c-449b-a837-b26342de8b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:02.746775 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed7cde-537c-449b-a837-b26342de8b59 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:02.747057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed7cde-537c-449b-a837-b26342de8b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:02.747330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed7cde-537c-449b-a837-b26342de8b59 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:02.748169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed7cde-537c-449b-a837-b26342de8b59 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:02.748479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed7cde-537c-449b-a837-b26342de8b59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:02.749916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed7cde-537c-449b-a837-b26342de8b59 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:02.769319 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 539/1069] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Jul 20 08:13:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:02.774546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed7cde-537c-449b-a837-b26342de8b59 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:02.777005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb66f96-5874-4d40-b723-9c9eee06f4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:02.777867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb66f96-5874-4d40-b723-9c9eee06f4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:02.778772 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb66f96-5874-4d40-b723-9c9eee06f4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:02.779085 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb66f96-5874-4d40-b723-9c9eee06f4b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:02.858140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb66f96-5874-4d40-b723-9c9eee06f4b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:02.861285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb66f96-5874-4d40-b723-9c9eee06f4b8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:02.868358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:02.890143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed7cde-537c-449b-a837-b26342de8b59 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:02.919052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:02.933998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce2dc5-6fc0-4150-8f01-793304f92cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:02.935081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce2dc5-6fc0-4150-8f01-793304f92cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:02.935549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce2dc5-6fc0-4150-8f01-793304f92cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:02.935824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce2dc5-6fc0-4150-8f01-793304f92cf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:02.947009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce2dc5-6fc0-4150-8f01-793304f92cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:02.948472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce2dc5-6fc0-4150-8f01-793304f92cf5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-61ce2dc5-6fc0-4150-8f01-793304f92cf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:13:02.968129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:02.974959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:02.975862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:13:02.975862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:13:02.975862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:13:02.975862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:02.975862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 540/1070] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:13:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:13:02.975862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6400e52-9400-4594-9ae8-09373f5dcc3c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:02.975862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6400e52-9400-4594-9ae8-09373f5dcc3c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:02.975862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6400e52-9400-4594-9ae8-09373f5dcc3c None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:02.975862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 541/1071] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:02.976415 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6400e52-9400-4594-9ae8-09373f5dcc3c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:02.979200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 531/1072] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:13:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 08:13:02.980499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a1fe3-77a5-4840-a989-71d5e11b455d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:02.980796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a1fe3-77a5-4840-a989-71d5e11b455d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:02.981076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a1fe3-77a5-4840-a989-71d5e11b455d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:02.997363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-be8a1fe3-77a5-4840-a989-71d5e11b455d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:02.998563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-be8a1fe3-77a5-4840-a989-71d5e11b455d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:03.080722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 532/1073] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:03.081938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a1fe3-77a5-4840-a989-71d5e11b455d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:03.108074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6400e52-9400-4594-9ae8-09373f5dcc3c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:03.142251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:03.147822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecfe44-91ca-493c-b40e-6efaab62b751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:03.148489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecfe44-91ca-493c-b40e-6efaab62b751 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:03.148781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecfe44-91ca-493c-b40e-6efaab62b751 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:03.149059 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecfe44-91ca-493c-b40e-6efaab62b751 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:03.149787 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecfe44-91ca-493c-b40e-6efaab62b751 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:03.150095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecfe44-91ca-493c-b40e-6efaab62b751 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:03.151504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecfe44-91ca-493c-b40e-6efaab62b751 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:03.163178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 542/1074] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:03.163998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecfe44-91ca-493c-b40e-6efaab62b751 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:03.175083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:03.198831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a1fe3-77a5-4840-a989-71d5e11b455d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:03.236347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55fa52-d64d-49ef-b1ac-9b29f26ef6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:03.237499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55fa52-d64d-49ef-b1ac-9b29f26ef6ab None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:03.237812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55fa52-d64d-49ef-b1ac-9b29f26ef6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:03.238106 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55fa52-d64d-49ef-b1ac-9b29f26ef6ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:03.249295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55fa52-d64d-49ef-b1ac-9b29f26ef6ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:03.251362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55fa52-d64d-49ef-b1ac-9b29f26ef6ab None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-fd55fa52-d64d-49ef-b1ac-9b29f26ef6ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:13:03.270993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:03.272054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:03.272938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:13:03.272938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:13:03.272938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:13:03.272938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:13:03.273155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecfe44-91ca-493c-b40e-6efaab62b751 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:03.273368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 533/1075] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:13:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:03.273368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55fa52-d64d-49ef-b1ac-9b29f26ef6ab None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:03.312005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06d9ad-bb07-4dba-be16-a824f691cbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:03.313557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06d9ad-bb07-4dba-be16-a824f691cbaf None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:03.314054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06d9ad-bb07-4dba-be16-a824f691cbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:03.314555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff06d9ad-bb07-4dba-be16-a824f691cbaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:03.327457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff06d9ad-bb07-4dba-be16-a824f691cbaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ff06d9ad-bb07-4dba-be16-a824f691cbaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:13:03.336328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:03.337473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:03.338497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 543/1076] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:13:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:03.339400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06d9ad-bb07-4dba-be16-a824f691cbaf None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:03.396549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55fa52-d64d-49ef-b1ac-9b29f26ef6ab None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:03.404528 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:03.429639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:03.439476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34547138-5288-4a8a-b7be-7e908a32ce95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:03.440295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34547138-5288-4a8a-b7be-7e908a32ce95 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:03.440587 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34547138-5288-4a8a-b7be-7e908a32ce95 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:03.440868 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34547138-5288-4a8a-b7be-7e908a32ce95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:03.448658 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34547138-5288-4a8a-b7be-7e908a32ce95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:03.450113 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34547138-5288-4a8a-b7be-7e908a32ce95 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-34547138-5288-4a8a-b7be-7e908a32ce95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:13:03.464802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:03.465924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:03.466985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:13:03.466985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:13:03.466985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:13:03.466985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:13:03.466985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:13:03.466985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:13:03.466985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:13:03.466985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:13:03.466985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 534/1077] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:13:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:03.467491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34547138-5288-4a8a-b7be-7e908a32ce95 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:03.472008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff06d9ad-bb07-4dba-be16-a824f691cbaf None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:03.514199 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d643d6a-889d-41c7-86b5-a3015ada99f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:03.515599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d643d6a-889d-41c7-86b5-a3015ada99f3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:03.516122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d643d6a-889d-41c7-86b5-a3015ada99f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:03.516810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d643d6a-889d-41c7-86b5-a3015ada99f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:03.517674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d643d6a-889d-41c7-86b5-a3015ada99f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:03.517946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d643d6a-889d-41c7-86b5-a3015ada99f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:03.519374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d643d6a-889d-41c7-86b5-a3015ada99f3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:03.532160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 544/1078] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:03.533196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d643d6a-889d-41c7-86b5-a3015ada99f3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:03.608438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34547138-5288-4a8a-b7be-7e908a32ce95 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:03.662184 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247e3ee-2e6c-43f9-998a-09f1d56c55eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:03.663003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247e3ee-2e6c-43f9-998a-09f1d56c55eb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:03.663408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247e3ee-2e6c-43f9-998a-09f1d56c55eb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:03.663803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247e3ee-2e6c-43f9-998a-09f1d56c55eb None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:03.664694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247e3ee-2e6c-43f9-998a-09f1d56c55eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:03.665124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247e3ee-2e6c-43f9-998a-09f1d56c55eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:03.666655 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247e3ee-2e6c-43f9-998a-09f1d56c55eb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:03.667885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d643d6a-889d-41c7-86b5-a3015ada99f3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:03.681654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 535/1079] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:13:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:03.690762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247e3ee-2e6c-43f9-998a-09f1d56c55eb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:03.701876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:03.735647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c95ba6-e327-418b-ba79-3f596d077bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:03.736975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c95ba6-e327-418b-ba79-3f596d077bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:03.737163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c95ba6-e327-418b-ba79-3f596d077bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:03.737424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c95ba6-e327-418b-ba79-3f596d077bf9 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:03.738177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c95ba6-e327-418b-ba79-3f596d077bf9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:03.738487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c95ba6-e327-418b-ba79-3f596d077bf9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:03.739980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c95ba6-e327-418b-ba79-3f596d077bf9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:03.756604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 545/1080] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:13:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:03.762823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c95ba6-e327-418b-ba79-3f596d077bf9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:03.786003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:03.847723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247e3ee-2e6c-43f9-998a-09f1d56c55eb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:03.883968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c01e2-44a8-4296-999d-1551b16c0f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:03.884843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c01e2-44a8-4296-999d-1551b16c0f54 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:03.885126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c01e2-44a8-4296-999d-1551b16c0f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:03.885373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c95ba6-e327-418b-ba79-3f596d077bf9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:03.885484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c01e2-44a8-4296-999d-1551b16c0f54 None admin] PATH_INFO: `/v3/users/35fb90211c9346a98851ccc1fcd4385a` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:03.886619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c01e2-44a8-4296-999d-1551b16c0f54 None admin] RBAC: Authorizing `identity:get_user(user_id=35fb90211c9346a98851ccc1fcd4385a)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:03.888063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c01e2-44a8-4296-999d-1551b16c0f54 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:03.889693 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 536/1081] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:03 2022] GET /identity/v3/users/35fb90211c9346a98851ccc1fcd4385a => generated 285 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:03.895912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c01e2-44a8-4296-999d-1551b16c0f54 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:03.925082 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bff56-1366-4a8e-b031-d4ac014d55e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:03.925895 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bff56-1366-4a8e-b031-d4ac014d55e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:03.926182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bff56-1366-4a8e-b031-d4ac014d55e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:03.926463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bff56-1366-4a8e-b031-d4ac014d55e8 None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/84f0da8709624834976d0662aeb02ea6/roles/6747431cfb4944ceadd7cb581c7a23f8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:03.928759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bff56-1366-4a8e-b031-d4ac014d55e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=84f0da8709624834976d0662aeb02ea6, role_id=6747431cfb4944ceadd7cb581c7a23f8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:03.930218 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bff56-1366-4a8e-b031-d4ac014d55e8 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:03.943661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 546/1082] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:13:03 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/84f0da8709624834976d0662aeb02ea6/roles/6747431cfb4944ceadd7cb581c7a23f8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:13:04.017954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c01e2-44a8-4296-999d-1551b16c0f54 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:04.041748 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:04.062619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ea568-02de-40ab-be41-e95cd04b8f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:04.063519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ea568-02de-40ab-be41-e95cd04b8f8c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:04.063800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ea568-02de-40ab-be41-e95cd04b8f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:04.064076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ea568-02de-40ab-be41-e95cd04b8f8c None admin] PATH_INFO: `/v3/projects/04e3033a1c75463fb773a9a8587eb50b` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:04.065413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ea568-02de-40ab-be41-e95cd04b8f8c None admin] RBAC: Authorizing `identity:get_project(project_id=04e3033a1c75463fb773a9a8587eb50b)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:04.066930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ea568-02de-40ab-be41-e95cd04b8f8c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:04.068817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 537/1083] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:03 2022] GET /identity/v3/projects/04e3033a1c75463fb773a9a8587eb50b => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:04.075465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bff56-1366-4a8e-b031-d4ac014d55e8 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:04.113609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:04.190618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bff56-1366-4a8e-b031-d4ac014d55e8 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:04.227287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826ff35-cf4a-4517-bb68-37648b48e63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:04.228187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826ff35-cf4a-4517-bb68-37648b48e63a None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:04.228473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826ff35-cf4a-4517-bb68-37648b48e63a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:04.228745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826ff35-cf4a-4517-bb68-37648b48e63a None admin] PATH_INFO: `/v3/projects/04e3033a1c75463fb773a9a8587eb50b/users/35fb90211c9346a98851ccc1fcd4385a/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:04.230743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826ff35-cf4a-4517-bb68-37648b48e63a None admin] RBAC: Authorizing `identity:create_grant(project_id=04e3033a1c75463fb773a9a8587eb50b, user_id=35fb90211c9346a98851ccc1fcd4385a, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:04.232200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826ff35-cf4a-4517-bb68-37648b48e63a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:04.246089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 547/1084] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:13:04 2022] PUT /identity/v3/projects/04e3033a1c75463fb773a9a8587eb50b/users/35fb90211c9346a98851ccc1fcd4385a/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:13:06.003203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45772bc-80b9-483c-bf9f-fb225f3165a2 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.003770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45772bc-80b9-483c-bf9f-fb225f3165a2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.003770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45772bc-80b9-483c-bf9f-fb225f3165a2 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.005261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 538/1085] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:06.010623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290dda3-cb9e-4909-9e6b-e22eca8947ce None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.011074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290dda3-cb9e-4909-9e6b-e22eca8947ce None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.011190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290dda3-cb9e-4909-9e6b-e22eca8947ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.032820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0290dda3-cb9e-4909-9e6b-e22eca8947ce None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:06.034022 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0290dda3-cb9e-4909-9e6b-e22eca8947ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:06.104759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 548/1086] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:06.129063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45772bc-80b9-483c-bf9f-fb225f3165a2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:06.232608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45772bc-80b9-483c-bf9f-fb225f3165a2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:06.268012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49edce-0c1e-432a-8c54-7338b56d8330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:06.268743 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49edce-0c1e-432a-8c54-7338b56d8330 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.269025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49edce-0c1e-432a-8c54-7338b56d8330 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.269301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49edce-0c1e-432a-8c54-7338b56d8330 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.279575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49edce-0c1e-432a-8c54-7338b56d8330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-6b49edce-0c1e-432a-8c54-7338b56d8330 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:13:06.288352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:06.289314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:13:06.290133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 539/1087] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Jul 20 08:13:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:06.295621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290dda3-cb9e-4909-9e6b-e22eca8947ce None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:06.321054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:06.332345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261602c-89ae-4ab5-b679-2ed13d7aff73 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.332636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261602c-89ae-4ab5-b679-2ed13d7aff73 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.332924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261602c-89ae-4ab5-b679-2ed13d7aff73 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.334470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 540/1088] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:13:06.339946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd977b3-5537-47f4-95c8-59359cd609d5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.340231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd977b3-5537-47f4-95c8-59359cd609d5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.340509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd977b3-5537-47f4-95c8-59359cd609d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.358400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1fd977b3-5537-47f4-95c8-59359cd609d5 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:06.360482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1fd977b3-5537-47f4-95c8-59359cd609d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:06.402978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290dda3-cb9e-4909-9e6b-e22eca8947ce None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:06.408393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:06.434423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 541/1089] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:13:06.440425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962733c-a777-463b-945e-1d39f644372c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:06.441163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962733c-a777-463b-945e-1d39f644372c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.441388 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962733c-a777-463b-945e-1d39f644372c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.441670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962733c-a777-463b-945e-1d39f644372c None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.442424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962733c-a777-463b-945e-1d39f644372c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:06.442773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962733c-a777-463b-945e-1d39f644372c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:06.444163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962733c-a777-463b-945e-1d39f644372c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:06.452682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 549/1090] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Jul 20 08:13:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:06.458947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd977b3-5537-47f4-95c8-59359cd609d5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:06.459155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962733c-a777-463b-945e-1d39f644372c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:06.554221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:06.568300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd977b3-5537-47f4-95c8-59359cd609d5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:06.571153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962733c-a777-463b-945e-1d39f644372c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:06.606830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206c874-6256-4053-abe7-b416c9c074a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:06.607642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c874-6256-4053-abe7-b416c9c074a9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.607923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c874-6256-4053-abe7-b416c9c074a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.608206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206c874-6256-4053-abe7-b416c9c074a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.608579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69fb36-0d81-462d-b607-f031e495a150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:06.609524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69fb36-0d81-462d-b607-f031e495a150 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.609926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69fb36-0d81-462d-b607-f031e495a150 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.610318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69fb36-0d81-462d-b607-f031e495a150 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.622150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69fb36-0d81-462d-b607-f031e495a150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:13:06.622341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206c874-6256-4053-abe7-b416c9c074a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:06.623812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206c874-6256-4053-abe7-b416c9c074a9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-ed69fb36-0d81-462d-b607-f031e495a150 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:13:06.629987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:06.631129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:06.632000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 550/1091] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:13:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:13:06.637476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69fb36-0d81-462d-b607-f031e495a150 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-8206c874-6256-4053-abe7-b416c9c074a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:13:06.644097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:06.645034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:06.645900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:13:06.645900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:13:06.645900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:13:06.645900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:13:06.645900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 542/1092] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:13:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:06.651144 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206c874-6256-4053-abe7-b416c9c074a9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:06.684413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:06.755728 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69fb36-0d81-462d-b607-f031e495a150 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:06.762885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206c874-6256-4053-abe7-b416c9c074a9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:06.794817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873543a2-4015-4695-b20b-dad2124c9c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:06.795487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873543a2-4015-4695-b20b-dad2124c9c1a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.795769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873543a2-4015-4695-b20b-dad2124c9c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.796046 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873543a2-4015-4695-b20b-dad2124c9c1a None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.796820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873543a2-4015-4695-b20b-dad2124c9c1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:06.797188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873543a2-4015-4695-b20b-dad2124c9c1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:06.798567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873543a2-4015-4695-b20b-dad2124c9c1a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:06.799496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556d93c-837b-4944-8f47-4186c4ba6693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:06.800209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556d93c-837b-4944-8f47-4186c4ba6693 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.800456 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556d93c-837b-4944-8f47-4186c4ba6693 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.800735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556d93c-837b-4944-8f47-4186c4ba6693 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.801464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556d93c-837b-4944-8f47-4186c4ba6693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:06.801774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556d93c-837b-4944-8f47-4186c4ba6693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:06.803175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556d93c-837b-4944-8f47-4186c4ba6693 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:06.807688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 551/1093] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:13:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:06.814161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873543a2-4015-4695-b20b-dad2124c9c1a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:06.820958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 543/1094] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:06.822012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556d93c-837b-4944-8f47-4186c4ba6693 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:06.823620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:06.921763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873543a2-4015-4695-b20b-dad2124c9c1a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:06.924229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:06.935273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556d93c-837b-4944-8f47-4186c4ba6693 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:06.961138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979336a-ae6b-484a-9fa3-7a83f82c3989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:06.962012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979336a-ae6b-484a-9fa3-7a83f82c3989 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.962306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979336a-ae6b-484a-9fa3-7a83f82c3989 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.962609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979336a-ae6b-484a-9fa3-7a83f82c3989 None admin] PATH_INFO: `/v3/users/35fb90211c9346a98851ccc1fcd4385a` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.963845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979336a-ae6b-484a-9fa3-7a83f82c3989 None admin] RBAC: Authorizing `identity:get_user(user_id=35fb90211c9346a98851ccc1fcd4385a)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:06.965300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a979336a-ae6b-484a-9fa3-7a83f82c3989 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:06.966995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 552/1095] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:06 2022] GET /identity/v3/users/35fb90211c9346a98851ccc1fcd4385a => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:06.970717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb4587-680a-4a89-ad1c-951d85c7f850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:06.971525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb4587-680a-4a89-ad1c-951d85c7f850 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:06.971808 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb4587-680a-4a89-ad1c-951d85c7f850 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:06.972134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb4587-680a-4a89-ad1c-951d85c7f850 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:06.973631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979336a-ae6b-484a-9fa3-7a83f82c3989 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:06.980385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb4587-680a-4a89-ad1c-951d85c7f850 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:06.981852 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb4587-680a-4a89-ad1c-951d85c7f850 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-76cb4587-680a-4a89-ad1c-951d85c7f850 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:13:06.997925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:06.999032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:07.000018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:07.000018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:07.000018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:13:07.000018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:13:07.000018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:13:07.000018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:13:07.000018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:13:07.000018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:13:07.000018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 08:13:07.000018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:13:07.000018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 544/1096] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Jul 20 08:13:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:07.005716 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb4587-680a-4a89-ad1c-951d85c7f850 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:07.059268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:07.088451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979336a-ae6b-484a-9fa3-7a83f82c3989 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:07.125027 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb4587-680a-4a89-ad1c-951d85c7f850 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:07.132537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590f3e3-28b6-44a6-8eec-2e31de4584f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:07.133693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590f3e3-28b6-44a6-8eec-2e31de4584f9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:07.134202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590f3e3-28b6-44a6-8eec-2e31de4584f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:07.134760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590f3e3-28b6-44a6-8eec-2e31de4584f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:07.162073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0393b-2b5c-43aa-b4ff-628b7e82d1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:07.162907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0393b-2b5c-43aa-b4ff-628b7e82d1eb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:07.163242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0393b-2b5c-43aa-b4ff-628b7e82d1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:07.163529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb0393b-2b5c-43aa-b4ff-628b7e82d1eb None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:07.164389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0393b-2b5c-43aa-b4ff-628b7e82d1eb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:07.164702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0393b-2b5c-43aa-b4ff-628b7e82d1eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:07.166159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb0393b-2b5c-43aa-b4ff-628b7e82d1eb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:07.188489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 545/1097] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Jul 20 08:13:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:07.195174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0393b-2b5c-43aa-b4ff-628b7e82d1eb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:07.207101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:07.228186 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590f3e3-28b6-44a6-8eec-2e31de4584f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:07.229640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590f3e3-28b6-44a6-8eec-2e31de4584f9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:07.311930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb0393b-2b5c-43aa-b4ff-628b7e82d1eb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:07.345317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:07.354816 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443f1eb-2756-4cc5-9a01-468625f86ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:07.355479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443f1eb-2756-4cc5-9a01-468625f86ded None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:07.355761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443f1eb-2756-4cc5-9a01-468625f86ded None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:07.356032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5443f1eb-2756-4cc5-9a01-468625f86ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:07.375743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 553/1098] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:13:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 08:13:07.381992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590f3e3-28b6-44a6-8eec-2e31de4584f9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:07.436152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443f1eb-2756-4cc5-9a01-468625f86ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:07.438166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5443f1eb-2756-4cc5-9a01-468625f86ded None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:07.449472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:07.513508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590f3e3-28b6-44a6-8eec-2e31de4584f9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:07.552385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9137f6-882f-42f9-927a-cd57af3212c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:07.553263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9137f6-882f-42f9-927a-cd57af3212c7 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:07.553554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9137f6-882f-42f9-927a-cd57af3212c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:07.553843 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9137f6-882f-42f9-927a-cd57af3212c7 None admin] PATH_INFO: `/v3/projects/04e3033a1c75463fb773a9a8587eb50b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:07.555121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9137f6-882f-42f9-927a-cd57af3212c7 None admin] RBAC: Authorizing `identity:get_project(project_id=04e3033a1c75463fb773a9a8587eb50b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:07.556590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9137f6-882f-42f9-927a-cd57af3212c7 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:07.558341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 554/1099] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:07 2022] GET /identity/v3/projects/04e3033a1c75463fb773a9a8587eb50b => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:07.559282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 546/1100] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:13:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 08:13:07.565833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9137f6-882f-42f9-927a-cd57af3212c7 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:07.566002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443f1eb-2756-4cc5-9a01-468625f86ded None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:07.617608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:07.683112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5443f1eb-2756-4cc5-9a01-468625f86ded None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:07.686515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9137f6-882f-42f9-927a-cd57af3212c7 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:07.695475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:07.734190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b427229-8426-4db6-af93-1bbc266e0df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:07.735530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ee452-666c-4ac7-a57a-7179647d00cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:07.735829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427229-8426-4db6-af93-1bbc266e0df0 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:07.736293 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427229-8426-4db6-af93-1bbc266e0df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:07.736822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee452-666c-4ac7-a57a-7179647d00cd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:07.737026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b427229-8426-4db6-af93-1bbc266e0df0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:07.737287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee452-666c-4ac7-a57a-7179647d00cd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:07.737736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ee452-666c-4ac7-a57a-7179647d00cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:07.738841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ee452-666c-4ac7-a57a-7179647d00cd None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=04e3033a1c75463fb773a9a8587eb50b, user.id=35fb90211c9346a98851ccc1fcd4385a) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:07.739155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ee452-666c-4ac7-a57a-7179647d00cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:07.740639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ee452-666c-4ac7-a57a-7179647d00cd None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:07.747564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 555/1101] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:13:07 2022] GET /identity/v3/role_assignments?user.id=35fb90211c9346a98851ccc1fcd4385a&scope.project.id=04e3033a1c75463fb773a9a8587eb50b&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:07.749243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b427229-8426-4db6-af93-1bbc266e0df0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:07.751425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b427229-8426-4db6-af93-1bbc266e0df0 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-4b427229-8426-4db6-af93-1bbc266e0df0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:13:07.771924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:07.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:07.773764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:13:07.773764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:13:07.773764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:13:07.773764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:13:07.773764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 547/1102] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:13:07.779900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ee452-666c-4ac7-a57a-7179647d00cd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:07.920542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ee452-666c-4ac7-a57a-7179647d00cd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:07.929628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:07.959565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c6f96-048e-45dc-b251-b274eb452fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:07.960238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c6f96-048e-45dc-b251-b274eb452fee None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:07.960525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c6f96-048e-45dc-b251-b274eb452fee None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:07.960812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c6f96-048e-45dc-b251-b274eb452fee None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:07.961605 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c6f96-048e-45dc-b251-b274eb452fee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:07.961911 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c6f96-048e-45dc-b251-b274eb452fee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:07.963356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c6f96-048e-45dc-b251-b274eb452fee None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:07.977258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 556/1103] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:07.983197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b427229-8426-4db6-af93-1bbc266e0df0 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:08.098036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b427229-8426-4db6-af93-1bbc266e0df0 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:08.136365 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6df00-d4b6-4099-8494-7afe3c951240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:08.137166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6df00-d4b6-4099-8494-7afe3c951240 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:08.137472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6df00-d4b6-4099-8494-7afe3c951240 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:08.137737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6df00-d4b6-4099-8494-7afe3c951240 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:08.145321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6df00-d4b6-4099-8494-7afe3c951240 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:08.146820 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6df00-d4b6-4099-8494-7afe3c951240 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-4ad6df00-d4b6-4099-8494-7afe3c951240 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:13:08.161247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:08.162454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:08.163360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:13:08.163360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:13:08.163360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:13:08.163360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:13:08.163360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:13:08.163360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:13:08.163360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:13:08.163360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:13:08.163360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 548/1104] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:13:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:08.168622 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c6f96-048e-45dc-b251-b274eb452fee None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:08.285538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c6f96-048e-45dc-b251-b274eb452fee None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:08.321442 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6379dd1-5281-4f6f-aa07-b8de2c3d70b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:08.322106 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6379dd1-5281-4f6f-aa07-b8de2c3d70b3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:08.322384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6379dd1-5281-4f6f-aa07-b8de2c3d70b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:08.322680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6379dd1-5281-4f6f-aa07-b8de2c3d70b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:08.323479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6379dd1-5281-4f6f-aa07-b8de2c3d70b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:08.323781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6379dd1-5281-4f6f-aa07-b8de2c3d70b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:08.325264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6379dd1-5281-4f6f-aa07-b8de2c3d70b3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:08.338685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 557/1105] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:13:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:08.345654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6df00-d4b6-4099-8494-7afe3c951240 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:08.373054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:08.461919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6df00-d4b6-4099-8494-7afe3c951240 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:08.499251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bf8eb-1189-4039-8b8c-2f303169e319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:08.500450 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf8eb-1189-4039-8b8c-2f303169e319 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:08.500751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf8eb-1189-4039-8b8c-2f303169e319 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:08.501024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf8eb-1189-4039-8b8c-2f303169e319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:08.501875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bf8eb-1189-4039-8b8c-2f303169e319 None admin] RBAC: Adding query filter params (role.id=6747431cfb4944ceadd7cb581c7a23f8, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=84f0da8709624834976d0662aeb02ea6) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:08.502189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bf8eb-1189-4039-8b8c-2f303169e319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:08.503711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bf8eb-1189-4039-8b8c-2f303169e319 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:08.510632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 549/1106] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:13:08 2022] GET /identity/v3/role_assignments?user.id=84f0da8709624834976d0662aeb02ea6&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=6747431cfb4944ceadd7cb581c7a23f8 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:09.869838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b5b66-99d1-45cd-b788-2aae2bd672fe None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:09.870423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b5b66-99d1-45cd-b788-2aae2bd672fe None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:09.870423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b5b66-99d1-45cd-b788-2aae2bd672fe None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:09.871797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 558/1107] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:09.877660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b030507-90b9-4aa4-9490-e91e88898397 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:09.878131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b030507-90b9-4aa4-9490-e91e88898397 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:09.878226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b030507-90b9-4aa4-9490-e91e88898397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:09.895920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1b030507-90b9-4aa4-9490-e91e88898397 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:09.897132 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1b030507-90b9-4aa4-9490-e91e88898397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:09.968375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 550/1108] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:09.993006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b5b66-99d1-45cd-b788-2aae2bd672fe None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:10.023308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:10.103830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b5b66-99d1-45cd-b788-2aae2bd672fe None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:10.138831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669ea0d-953c-4e6e-99fc-956acb5d9868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:10.139544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669ea0d-953c-4e6e-99fc-956acb5d9868 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:10.139825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669ea0d-953c-4e6e-99fc-956acb5d9868 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:10.140100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669ea0d-953c-4e6e-99fc-956acb5d9868 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:10.141936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669ea0d-953c-4e6e-99fc-956acb5d9868 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:10.143412 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669ea0d-953c-4e6e-99fc-956acb5d9868 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:10.145039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 559/1109] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:13:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:10.151249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b030507-90b9-4aa4-9490-e91e88898397 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:10.166476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:10.251560 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b030507-90b9-4aa4-9490-e91e88898397 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:10.285357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb593f-65b3-4b9d-b4b3-ae5e4bbab098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:10.286027 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb593f-65b3-4b9d-b4b3-ae5e4bbab098 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:10.286318 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb593f-65b3-4b9d-b4b3-ae5e4bbab098 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:10.286613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb593f-65b3-4b9d-b4b3-ae5e4bbab098 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:10.287339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb593f-65b3-4b9d-b4b3-ae5e4bbab098 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:10.288793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb593f-65b3-4b9d-b4b3-ae5e4bbab098 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:10.309280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 551/1110] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:13:10 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:13:12.388386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768b643-afe7-427b-84f8-99f849df6423 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:12.388960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768b643-afe7-427b-84f8-99f849df6423 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:12.388960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768b643-afe7-427b-84f8-99f849df6423 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:12.390316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 560/1111] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:12.396065 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ed67a-a37a-483f-82fd-8f656df9943a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:12.396488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ed67a-a37a-483f-82fd-8f656df9943a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:12.396634 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ed67a-a37a-483f-82fd-8f656df9943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:12.414179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-593ed67a-a37a-483f-82fd-8f656df9943a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:12.415393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-593ed67a-a37a-483f-82fd-8f656df9943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:12.477812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:12.483367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 552/1112] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:12.507859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768b643-afe7-427b-84f8-99f849df6423 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:12.602524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:12.611590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768b643-afe7-427b-84f8-99f849df6423 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:12.648653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f0314-2bbe-4f3f-8f9d-4e50ef0aa3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:12.649420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f0314-2bbe-4f3f-8f9d-4e50ef0aa3ba None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:12.649700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f0314-2bbe-4f3f-8f9d-4e50ef0aa3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:12.649974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f0314-2bbe-4f3f-8f9d-4e50ef0aa3ba None admin] PATH_INFO: `/v3/roles/1d508b4effb546039e87dd273933af16` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:12.651463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f0314-2bbe-4f3f-8f9d-4e50ef0aa3ba None admin] RBAC: Authorizing `identity:get_role(role_id=1d508b4effb546039e87dd273933af16)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:12.652844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f0314-2bbe-4f3f-8f9d-4e50ef0aa3ba None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:12.654219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 561/1113] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:12 2022] GET /identity/v3/roles/1d508b4effb546039e87dd273933af16 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:12.660195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ed67a-a37a-483f-82fd-8f656df9943a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:12.773393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ed67a-a37a-483f-82fd-8f656df9943a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:12.807879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df610b88-3941-4f6d-9119-d0ee32d1890c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:12.808762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df610b88-3941-4f6d-9119-d0ee32d1890c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:12.809051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df610b88-3941-4f6d-9119-d0ee32d1890c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:12.809332 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df610b88-3941-4f6d-9119-d0ee32d1890c None admin] PATH_INFO: `/v3/users/3f54e76b9a2d49ef9dc9ee770c8051cd` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:12.823458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df610b88-3941-4f6d-9119-d0ee32d1890c None admin] RBAC: Authorizing `identity:get_user(user_id=3f54e76b9a2d49ef9dc9ee770c8051cd)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:12.824910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df610b88-3941-4f6d-9119-d0ee32d1890c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:12.826595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 553/1114] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:12 2022] GET /identity/v3/users/3f54e76b9a2d49ef9dc9ee770c8051cd => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:12.832529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f0314-2bbe-4f3f-8f9d-4e50ef0aa3ba None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:12.935784 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f0314-2bbe-4f3f-8f9d-4e50ef0aa3ba None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:12.970957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffc961-c239-4461-aaf7-2f252409d905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:12.971602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffc961-c239-4461-aaf7-2f252409d905 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:12.971887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffc961-c239-4461-aaf7-2f252409d905 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:12.972163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ffc961-c239-4461-aaf7-2f252409d905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:13.003366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:13.042709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffc961-c239-4461-aaf7-2f252409d905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:13.044182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ffc961-c239-4461-aaf7-2f252409d905 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:13.143201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 562/1115] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:13:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:13:13.149187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df610b88-3941-4f6d-9119-d0ee32d1890c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:13.207951 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:13.250688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df610b88-3941-4f6d-9119-d0ee32d1890c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:13.285076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0538e7e4-9701-4d4f-aee0-71e7ac042b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:13.285931 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538e7e4-9701-4d4f-aee0-71e7ac042b54 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:13.286215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538e7e4-9701-4d4f-aee0-71e7ac042b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:13.286494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0538e7e4-9701-4d4f-aee0-71e7ac042b54 None admin] PATH_INFO: `/v3/projects/04e3033a1c75463fb773a9a8587eb50b` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:13.287700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538e7e4-9701-4d4f-aee0-71e7ac042b54 None admin] RBAC: Authorizing `identity:get_project(project_id=04e3033a1c75463fb773a9a8587eb50b)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:13.289129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0538e7e4-9701-4d4f-aee0-71e7ac042b54 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:13.290859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 554/1116] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:13 2022] GET /identity/v3/projects/04e3033a1c75463fb773a9a8587eb50b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:13.297205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffc961-c239-4461-aaf7-2f252409d905 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:13.358758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:13.399559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ffc961-c239-4461-aaf7-2f252409d905 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:13.434093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d523a-beb8-409c-963b-1845eab7ffb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:13.434782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d523a-beb8-409c-963b-1845eab7ffb9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:13.435070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d523a-beb8-409c-963b-1845eab7ffb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:13.435346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d523a-beb8-409c-963b-1845eab7ffb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:13.436178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d523a-beb8-409c-963b-1845eab7ffb9 None admin] RBAC: Adding query filter params (role.id=1d508b4effb546039e87dd273933af16, scope.project.id=04e3033a1c75463fb773a9a8587eb50b, user.id=3f54e76b9a2d49ef9dc9ee770c8051cd) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:13.436488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d523a-beb8-409c-963b-1845eab7ffb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:13.437975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d523a-beb8-409c-963b-1845eab7ffb9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:13.443680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 563/1117] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:13:13 2022] GET /identity/v3/role_assignments?user.id=3f54e76b9a2d49ef9dc9ee770c8051cd&scope.project.id=04e3033a1c75463fb773a9a8587eb50b&role.id=1d508b4effb546039e87dd273933af16 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:15.487432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b968df4-0970-45c0-8d82-298d6e9dd908 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:15.488018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b968df4-0970-45c0-8d82-298d6e9dd908 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:15.488018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b968df4-0970-45c0-8d82-298d6e9dd908 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:15.489358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 555/1118] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:15.494803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850a4d5-32d9-452b-9c66-746a041e31a8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:15.495215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850a4d5-32d9-452b-9c66-746a041e31a8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:15.495361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850a4d5-32d9-452b-9c66-746a041e31a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:15.512379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c850a4d5-32d9-452b-9c66-746a041e31a8 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:15.513574 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c850a4d5-32d9-452b-9c66-746a041e31a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:15.583592 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 564/1119] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:13:15.607423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b968df4-0970-45c0-8d82-298d6e9dd908 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:15.714325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b968df4-0970-45c0-8d82-298d6e9dd908 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:15.741271 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:15.750449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4812d-903f-4d26-8517-da322c796ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:15.751365 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4812d-903f-4d26-8517-da322c796ffb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:15.751651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4812d-903f-4d26-8517-da322c796ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:15.751929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4812d-903f-4d26-8517-da322c796ffb None admin] PATH_INFO: `/v3/roles/1d508b4effb546039e87dd273933af16` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:15.753409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4812d-903f-4d26-8517-da322c796ffb None admin] RBAC: Authorizing `identity:get_role(role_id=1d508b4effb546039e87dd273933af16)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:15.754818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4812d-903f-4d26-8517-da322c796ffb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:15.756198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 556/1120] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:15 2022] GET /identity/v3/roles/1d508b4effb546039e87dd273933af16 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:15.762412 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850a4d5-32d9-452b-9c66-746a041e31a8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:15.823355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:15.871801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850a4d5-32d9-452b-9c66-746a041e31a8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:15.908012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db2aa9-b67e-4bb4-9b1f-50e552d229ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:15.908880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db2aa9-b67e-4bb4-9b1f-50e552d229ee None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:15.909162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db2aa9-b67e-4bb4-9b1f-50e552d229ee None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:15.909444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db2aa9-b67e-4bb4-9b1f-50e552d229ee None admin] PATH_INFO: `/v3/users/3f54e76b9a2d49ef9dc9ee770c8051cd` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:15.910697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db2aa9-b67e-4bb4-9b1f-50e552d229ee None admin] RBAC: Authorizing `identity:get_user(user_id=3f54e76b9a2d49ef9dc9ee770c8051cd)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:15.912162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db2aa9-b67e-4bb4-9b1f-50e552d229ee None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:15.913848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 565/1121] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:15 2022] GET /identity/v3/users/3f54e76b9a2d49ef9dc9ee770c8051cd => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:15.919952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4812d-903f-4d26-8517-da322c796ffb None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:16.027844 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4812d-903f-4d26-8517-da322c796ffb None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:16.064371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ca0b7-bdbc-4eb7-a5c8-26d88364d86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:16.065229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ca0b7-bdbc-4eb7-a5c8-26d88364d86d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:16.065515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ca0b7-bdbc-4eb7-a5c8-26d88364d86d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:16.065793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ca0b7-bdbc-4eb7-a5c8-26d88364d86d None admin] PATH_INFO: `/v3/projects/04e3033a1c75463fb773a9a8587eb50b` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:16.067020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ca0b7-bdbc-4eb7-a5c8-26d88364d86d None admin] RBAC: Authorizing `identity:get_project(project_id=04e3033a1c75463fb773a9a8587eb50b)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:16.068462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ca0b7-bdbc-4eb7-a5c8-26d88364d86d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:16.070187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 557/1122] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:15 2022] GET /identity/v3/projects/04e3033a1c75463fb773a9a8587eb50b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:16.076865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db2aa9-b67e-4bb4-9b1f-50e552d229ee None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:16.186942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db2aa9-b67e-4bb4-9b1f-50e552d229ee None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:16.212428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:16.224142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c3d99-7f74-49af-b22b-4447e95137e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:16.225039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c3d99-7f74-49af-b22b-4447e95137e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:16.225323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c3d99-7f74-49af-b22b-4447e95137e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:16.225602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c3d99-7f74-49af-b22b-4447e95137e4 None admin] PATH_INFO: `/v3/projects/04e3033a1c75463fb773a9a8587eb50b/users/3f54e76b9a2d49ef9dc9ee770c8051cd/roles/1d508b4effb546039e87dd273933af16` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:16.228132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c3d99-7f74-49af-b22b-4447e95137e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=04e3033a1c75463fb773a9a8587eb50b, user_id=3f54e76b9a2d49ef9dc9ee770c8051cd, role_id=1d508b4effb546039e87dd273933af16)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:16.229596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c3d99-7f74-49af-b22b-4447e95137e4 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:16.244195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 566/1123] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:13:16 2022] PUT /identity/v3/projects/04e3033a1c75463fb773a9a8587eb50b/users/3f54e76b9a2d49ef9dc9ee770c8051cd/roles/1d508b4effb546039e87dd273933af16 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:13:18.416906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56418-7141-46cb-9a82-7e8a09cba22a None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:18.417546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56418-7141-46cb-9a82-7e8a09cba22a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:18.417546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56418-7141-46cb-9a82-7e8a09cba22a None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:18.418864 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 558/1124] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:18.426571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebc159-68a4-465d-8ce2-0efebd15cdd8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:18.427162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebc159-68a4-465d-8ce2-0efebd15cdd8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:18.427162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebc159-68a4-465d-8ce2-0efebd15cdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:18.445893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-16ebc159-68a4-465d-8ce2-0efebd15cdd8 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:18.447152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-16ebc159-68a4-465d-8ce2-0efebd15cdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:18.528474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 567/1125] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:18.553539 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56418-7141-46cb-9a82-7e8a09cba22a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:18.624218 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:18.673700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56418-7141-46cb-9a82-7e8a09cba22a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:18.720127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966ca0-b3d4-47a5-a665-7dcfdb4f2d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:18.721199 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966ca0-b3d4-47a5-a665-7dcfdb4f2d36 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:18.721566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966ca0-b3d4-47a5-a665-7dcfdb4f2d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:18.721877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed966ca0-b3d4-47a5-a665-7dcfdb4f2d36 None admin] PATH_INFO: `/v3/roles/1d508b4effb546039e87dd273933af16` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:18.723657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed966ca0-b3d4-47a5-a665-7dcfdb4f2d36 None admin] RBAC: Authorizing `identity:get_role(role_id=1d508b4effb546039e87dd273933af16)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:18.725091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed966ca0-b3d4-47a5-a665-7dcfdb4f2d36 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:18.726563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 559/1126] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:18 2022] GET /identity/v3/roles/1d508b4effb546039e87dd273933af16 => generated 223 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:18.733171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebc159-68a4-465d-8ce2-0efebd15cdd8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:18.867988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ebc159-68a4-465d-8ce2-0efebd15cdd8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:18.886508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:18.916596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053a177-4679-4a42-aa30-a733f65caf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:18.917622 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053a177-4679-4a42-aa30-a733f65caf12 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:18.917923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053a177-4679-4a42-aa30-a733f65caf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:18.918322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053a177-4679-4a42-aa30-a733f65caf12 None admin] PATH_INFO: `/v3/users/3f54e76b9a2d49ef9dc9ee770c8051cd` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:18.919650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053a177-4679-4a42-aa30-a733f65caf12 None admin] RBAC: Authorizing `identity:get_user(user_id=3f54e76b9a2d49ef9dc9ee770c8051cd)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:18.921388 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053a177-4679-4a42-aa30-a733f65caf12 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:18.924588 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 568/1127] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:18 2022] GET /identity/v3/users/3f54e76b9a2d49ef9dc9ee770c8051cd => generated 286 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:18.930840 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966ca0-b3d4-47a5-a665-7dcfdb4f2d36 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:19.056333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed966ca0-b3d4-47a5-a665-7dcfdb4f2d36 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:19.093086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:19.100827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57cd48-cd8a-4924-b933-703280f4eb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:19.101644 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57cd48-cd8a-4924-b933-703280f4eb7c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:19.101943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57cd48-cd8a-4924-b933-703280f4eb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:19.102225 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57cd48-cd8a-4924-b933-703280f4eb7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:19.186478 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57cd48-cd8a-4924-b933-703280f4eb7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:19.189040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57cd48-cd8a-4924-b933-703280f4eb7c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:19.303498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 560/1128] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:13:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 08:13:19.309695 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053a177-4679-4a42-aa30-a733f65caf12 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:19.424388 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053a177-4679-4a42-aa30-a733f65caf12 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:19.463873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdcc690-fff8-4e80-a75b-acfeafbcffa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:19.464675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcc690-fff8-4e80-a75b-acfeafbcffa6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:19.464970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcc690-fff8-4e80-a75b-acfeafbcffa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:19.465247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdcc690-fff8-4e80-a75b-acfeafbcffa6 None admin] PATH_INFO: `/v3/projects/04e3033a1c75463fb773a9a8587eb50b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:19.466632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcc690-fff8-4e80-a75b-acfeafbcffa6 None admin] RBAC: Authorizing `identity:get_project(project_id=04e3033a1c75463fb773a9a8587eb50b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:19.468098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdcc690-fff8-4e80-a75b-acfeafbcffa6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:19.470528 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 569/1129] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:19 2022] GET /identity/v3/projects/04e3033a1c75463fb773a9a8587eb50b => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:19.477007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57cd48-cd8a-4924-b933-703280f4eb7c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:19.509687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:19.596128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57cd48-cd8a-4924-b933-703280f4eb7c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:19.633175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38829abb-db81-4044-a637-be4ecdce8adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:19.633854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38829abb-db81-4044-a637-be4ecdce8adf None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:19.634147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38829abb-db81-4044-a637-be4ecdce8adf None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:19.634466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38829abb-db81-4044-a637-be4ecdce8adf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:19.635328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38829abb-db81-4044-a637-be4ecdce8adf None admin] RBAC: Adding query filter params (role.id=1d508b4effb546039e87dd273933af16, scope.project.id=04e3033a1c75463fb773a9a8587eb50b, user.id=3f54e76b9a2d49ef9dc9ee770c8051cd) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:19.635653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38829abb-db81-4044-a637-be4ecdce8adf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:19.637157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38829abb-db81-4044-a637-be4ecdce8adf None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:19.644956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 561/1130] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:13:19 2022] GET /identity/v3/role_assignments?user.id=3f54e76b9a2d49ef9dc9ee770c8051cd&scope.project.id=04e3033a1c75463fb773a9a8587eb50b&role.id=1d508b4effb546039e87dd273933af16 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:21.800862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840b93d-f822-49a0-b2bf-d84d766ca4a5 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:21.801468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840b93d-f822-49a0-b2bf-d84d766ca4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:21.801468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840b93d-f822-49a0-b2bf-d84d766ca4a5 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:21.802860 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 570/1131] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:21.808426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba8b5f-3901-4a75-a90c-70a1693f2abd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:21.808940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba8b5f-3901-4a75-a90c-70a1693f2abd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:21.809072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba8b5f-3901-4a75-a90c-70a1693f2abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:21.826354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a6ba8b5f-3901-4a75-a90c-70a1693f2abd None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:21.827593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a6ba8b5f-3901-4a75-a90c-70a1693f2abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:21.896084 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 562/1132] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:21.920225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840b93d-f822-49a0-b2bf-d84d766ca4a5 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:21.978804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:22.039586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840b93d-f822-49a0-b2bf-d84d766ca4a5 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:22.079762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b9fb9-a806-432c-a6be-e42d6ab7ed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:22.080496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9fb9-a806-432c-a6be-e42d6ab7ed22 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:22.080783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9fb9-a806-432c-a6be-e42d6ab7ed22 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:22.081091 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9fb9-a806-432c-a6be-e42d6ab7ed22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:22.082381 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b9fb9-a806-432c-a6be-e42d6ab7ed22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:22.083909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b9fb9-a806-432c-a6be-e42d6ab7ed22 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:22.085608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 571/1133] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:13:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:22.091697 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba8b5f-3901-4a75-a90c-70a1693f2abd None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:22.120972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:22.202807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba8b5f-3901-4a75-a90c-70a1693f2abd None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:22.241605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319903e-83ac-4e30-9eb2-c6d9427b8201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:22.242283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319903e-83ac-4e30-9eb2-c6d9427b8201 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:22.242572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319903e-83ac-4e30-9eb2-c6d9427b8201 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:22.242983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319903e-83ac-4e30-9eb2-c6d9427b8201 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:22.243714 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319903e-83ac-4e30-9eb2-c6d9427b8201 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:22.245180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8319903e-83ac-4e30-9eb2-c6d9427b8201 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:22.272393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 563/1134] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:13:22 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:13:24.456651 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad9699-658e-483b-bc71-55463007b49f None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:24.457250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad9699-658e-483b-bc71-55463007b49f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:24.457250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad9699-658e-483b-bc71-55463007b49f None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:24.458603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 572/1135] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:24.464984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbd8ed-d77e-4445-aba4-6a66b22afba5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:24.465589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbd8ed-d77e-4445-aba4-6a66b22afba5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:24.465589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cbd8ed-d77e-4445-aba4-6a66b22afba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:24.485030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-27cbd8ed-d77e-4445-aba4-6a66b22afba5 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:24.486680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-27cbd8ed-d77e-4445-aba4-6a66b22afba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:24.573578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 564/1136] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:24.598956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad9699-658e-483b-bc71-55463007b49f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:24.727606 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ad9699-658e-483b-bc71-55463007b49f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:24.752517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:24.771081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e054d422-773a-45e6-bbe9-7d34e848b59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:24.771883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054d422-773a-45e6-bbe9-7d34e848b59d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:24.772175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054d422-773a-45e6-bbe9-7d34e848b59d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:24.772480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e054d422-773a-45e6-bbe9-7d34e848b59d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:24.773693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054d422-773a-45e6-bbe9-7d34e848b59d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:24.775232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e054d422-773a-45e6-bbe9-7d34e848b59d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:24.776894 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 573/1137] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:13:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:24.782990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbd8ed-d77e-4445-aba4-6a66b22afba5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:24.791602 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:24.897354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cbd8ed-d77e-4445-aba4-6a66b22afba5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:24.942457 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573893f-681a-428b-be2b-25877c6a9293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:24.943173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573893f-681a-428b-be2b-25877c6a9293 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:24.943459 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573893f-681a-428b-be2b-25877c6a9293 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:24.943737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573893f-681a-428b-be2b-25877c6a9293 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:24.944477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573893f-681a-428b-be2b-25877c6a9293 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:24.945948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6573893f-681a-428b-be2b-25877c6a9293 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:24.966809 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 565/1138] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:13:24 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:13:27.158380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7828d2e-b654-498e-9c34-8d1711e2596c None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:27.158991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7828d2e-b654-498e-9c34-8d1711e2596c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:27.158991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7828d2e-b654-498e-9c34-8d1711e2596c None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:27.160331 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 574/1139] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:27.165593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35ddb3-4eea-49a1-a0a6-6c399846526b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:27.166039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35ddb3-4eea-49a1-a0a6-6c399846526b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:27.166163 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da35ddb3-4eea-49a1-a0a6-6c399846526b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:27.183298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-da35ddb3-4eea-49a1-a0a6-6c399846526b None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:27.184501 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-da35ddb3-4eea-49a1-a0a6-6c399846526b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:27.242254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:27.254134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 566/1140] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:27.278383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7828d2e-b654-498e-9c34-8d1711e2596c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:27.383386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7828d2e-b654-498e-9c34-8d1711e2596c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:27.417755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a4f73-a124-4854-8d1b-33e0d3b95d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:27.418547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a4f73-a124-4854-8d1b-33e0d3b95d4f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:27.418868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a4f73-a124-4854-8d1b-33e0d3b95d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:27.419145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a4f73-a124-4854-8d1b-33e0d3b95d4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:27.429706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a4f73-a124-4854-8d1b-33e0d3b95d4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-024a4f73-a124-4854-8d1b-33e0d3b95d4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:13:27.438243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:27.439428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:27.440523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 575/1141] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:13:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:27.445688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35ddb3-4eea-49a1-a0a6-6c399846526b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:27.547692 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da35ddb3-4eea-49a1-a0a6-6c399846526b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:27.587192 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0353a-5c67-4a2f-8017-f3f3a1769ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:27.587871 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0353a-5c67-4a2f-8017-f3f3a1769ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:27.588156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0353a-5c67-4a2f-8017-f3f3a1769ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:27.588434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f0353a-5c67-4a2f-8017-f3f3a1769ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:27.589186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0353a-5c67-4a2f-8017-f3f3a1769ab8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:27.589501 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0353a-5c67-4a2f-8017-f3f3a1769ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:27.590908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f0353a-5c67-4a2f-8017-f3f3a1769ab8 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:27.599336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 567/1142] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:13:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:27.605219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a4f73-a124-4854-8d1b-33e0d3b95d4f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:27.652640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:27.711501 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a4f73-a124-4854-8d1b-33e0d3b95d4f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:27.746019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364307d-c1ad-40ca-8f01-107d1e500259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:27.746920 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364307d-c1ad-40ca-8f01-107d1e500259 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:27.747208 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364307d-c1ad-40ca-8f01-107d1e500259 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:27.747485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364307d-c1ad-40ca-8f01-107d1e500259 None admin] PATH_INFO: `/v3/users/eee9195611f44c86b6e6d0b3483262c6` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:27.762225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364307d-c1ad-40ca-8f01-107d1e500259 None admin] RBAC: Authorizing `identity:get_user(user_id=eee9195611f44c86b6e6d0b3483262c6)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:27.763732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364307d-c1ad-40ca-8f01-107d1e500259 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:27.765478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 576/1143] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:27 2022] GET /identity/v3/users/eee9195611f44c86b6e6d0b3483262c6 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:27.771290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0353a-5c67-4a2f-8017-f3f3a1769ab8 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:27.837861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:27.886929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f0353a-5c67-4a2f-8017-f3f3a1769ab8 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:27.923178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22eb48-cd51-4eeb-bb68-b0b095678e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:27.923885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22eb48-cd51-4eeb-bb68-b0b095678e77 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:27.924168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22eb48-cd51-4eeb-bb68-b0b095678e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:27.924443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22eb48-cd51-4eeb-bb68-b0b095678e77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:27.996799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22eb48-cd51-4eeb-bb68-b0b095678e77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:27.998261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22eb48-cd51-4eeb-bb68-b0b095678e77 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:28.054509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:28.100473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 568/1144] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:13:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:13:28.108272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364307d-c1ad-40ca-8f01-107d1e500259 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:28.210292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364307d-c1ad-40ca-8f01-107d1e500259 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:28.212674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:28.245393 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7b993-2c4b-42a4-bb4d-886eaa7b3ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:28.246428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7b993-2c4b-42a4-bb4d-886eaa7b3ada None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:28.246664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7b993-2c4b-42a4-bb4d-886eaa7b3ada None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:28.246842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7b993-2c4b-42a4-bb4d-886eaa7b3ada None admin] PATH_INFO: `/v3/projects/1c410de37660487783d03a96e667cd7d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:28.248138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7b993-2c4b-42a4-bb4d-886eaa7b3ada None admin] RBAC: Authorizing `identity:get_project(project_id=1c410de37660487783d03a96e667cd7d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:28.249803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7b993-2c4b-42a4-bb4d-886eaa7b3ada None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:28.252184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 577/1145] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:28 2022] GET /identity/v3/projects/1c410de37660487783d03a96e667cd7d => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:28.258759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22eb48-cd51-4eeb-bb68-b0b095678e77 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:28.360494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22eb48-cd51-4eeb-bb68-b0b095678e77 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:28.400702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93963b0-0f3c-422c-88e6-60d9b06bfea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:28.401361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93963b0-0f3c-422c-88e6-60d9b06bfea2 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:28.401656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93963b0-0f3c-422c-88e6-60d9b06bfea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:28.401934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93963b0-0f3c-422c-88e6-60d9b06bfea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:28.402800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93963b0-0f3c-422c-88e6-60d9b06bfea2 None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=1c410de37660487783d03a96e667cd7d, user.id=eee9195611f44c86b6e6d0b3483262c6) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:28.403120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93963b0-0f3c-422c-88e6-60d9b06bfea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:28.404565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93963b0-0f3c-422c-88e6-60d9b06bfea2 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:28.410271 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 569/1146] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:13:28 2022] GET /identity/v3/role_assignments?user.id=eee9195611f44c86b6e6d0b3483262c6&scope.project.id=1c410de37660487783d03a96e667cd7d&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:30.460451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bad7f-f0dd-4249-963d-4bb9386a4cd4 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:30.461017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bad7f-f0dd-4249-963d-4bb9386a4cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:30.461017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bad7f-f0dd-4249-963d-4bb9386a4cd4 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:30.462363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 578/1147] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:30.467622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dda6e-3bfb-4c04-957f-ffe23abfe799 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:30.468066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dda6e-3bfb-4c04-957f-ffe23abfe799 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:30.468193 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4dda6e-3bfb-4c04-957f-ffe23abfe799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:30.485489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3c4dda6e-3bfb-4c04-957f-ffe23abfe799 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:30.486724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3c4dda6e-3bfb-4c04-957f-ffe23abfe799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:30.501113 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:30.556594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 570/1148] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:13:30.579817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bad7f-f0dd-4249-963d-4bb9386a4cd4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:30.686723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bad7f-f0dd-4249-963d-4bb9386a4cd4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:30.722986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d5c89-30ce-4aa0-93e6-f5ac251e2821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:30.723689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d5c89-30ce-4aa0-93e6-f5ac251e2821 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:30.723973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d5c89-30ce-4aa0-93e6-f5ac251e2821 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:30.724247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d5c89-30ce-4aa0-93e6-f5ac251e2821 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:30.734354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d5c89-30ce-4aa0-93e6-f5ac251e2821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-067d5c89-30ce-4aa0-93e6-f5ac251e2821 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:13:30.742118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:30.743130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:30.744007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 579/1149] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:13:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:30.749417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4dda6e-3bfb-4c04-957f-ffe23abfe799 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:30.851085 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4dda6e-3bfb-4c04-957f-ffe23abfe799 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:30.886209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b6439-3122-4db6-9a6a-e9ec5f9c38ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:30.886912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b6439-3122-4db6-9a6a-e9ec5f9c38ba None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:30.887206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b6439-3122-4db6-9a6a-e9ec5f9c38ba None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:30.887485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b6439-3122-4db6-9a6a-e9ec5f9c38ba None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:30.888229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b6439-3122-4db6-9a6a-e9ec5f9c38ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:30.888541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b6439-3122-4db6-9a6a-e9ec5f9c38ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:30.889915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b6439-3122-4db6-9a6a-e9ec5f9c38ba None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:30.898908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 571/1150] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:13:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:30.904978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d5c89-30ce-4aa0-93e6-f5ac251e2821 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:30.951727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:31.023198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d5c89-30ce-4aa0-93e6-f5ac251e2821 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:31.057230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97508d-4075-453d-8771-ce82e93230d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:31.058121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97508d-4075-453d-8771-ce82e93230d6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:31.058400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97508d-4075-453d-8771-ce82e93230d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:31.058702 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97508d-4075-453d-8771-ce82e93230d6 None admin] PATH_INFO: `/v3/users/eee9195611f44c86b6e6d0b3483262c6` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:31.059918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97508d-4075-453d-8771-ce82e93230d6 None admin] RBAC: Authorizing `identity:get_user(user_id=eee9195611f44c86b6e6d0b3483262c6)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:31.061380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97508d-4075-453d-8771-ce82e93230d6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:31.063828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 580/1151] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:30 2022] GET /identity/v3/users/eee9195611f44c86b6e6d0b3483262c6 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:31.070363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b6439-3122-4db6-9a6a-e9ec5f9c38ba None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:31.087201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:31.180056 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b6439-3122-4db6-9a6a-e9ec5f9c38ba None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:31.214361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9f0cd-5caa-44a2-a0c7-544f96bc95e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:31.215331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f0cd-5caa-44a2-a0c7-544f96bc95e5 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:31.215687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f0cd-5caa-44a2-a0c7-544f96bc95e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:31.215994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9f0cd-5caa-44a2-a0c7-544f96bc95e5 None admin] PATH_INFO: `/v3/projects/1c410de37660487783d03a96e667cd7d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:31.217243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9f0cd-5caa-44a2-a0c7-544f96bc95e5 None admin] RBAC: Authorizing `identity:get_project(project_id=1c410de37660487783d03a96e667cd7d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:31.219299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9f0cd-5caa-44a2-a0c7-544f96bc95e5 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:31.221008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 572/1152] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:31 2022] GET /identity/v3/projects/1c410de37660487783d03a96e667cd7d => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:31.227068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97508d-4075-453d-8771-ce82e93230d6 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:31.325303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:31.329013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97508d-4075-453d-8771-ce82e93230d6 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:31.365478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc1c85-4e94-4023-8c08-6d858964fcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:31.366363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc1c85-4e94-4023-8c08-6d858964fcec None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:31.366676 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc1c85-4e94-4023-8c08-6d858964fcec None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:31.366952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc1c85-4e94-4023-8c08-6d858964fcec None admin] PATH_INFO: `/v3/projects/1c410de37660487783d03a96e667cd7d/users/eee9195611f44c86b6e6d0b3483262c6/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:31.368949 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc1c85-4e94-4023-8c08-6d858964fcec None admin] RBAC: Authorizing `identity:create_grant(project_id=1c410de37660487783d03a96e667cd7d, user_id=eee9195611f44c86b6e6d0b3483262c6, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:31.370397 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc1c85-4e94-4023-8c08-6d858964fcec None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:31.384139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 581/1153] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:13:31 2022] PUT /identity/v3/projects/1c410de37660487783d03a96e667cd7d/users/eee9195611f44c86b6e6d0b3483262c6/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:13:33.472896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21cd1a-223e-419f-a3c7-f2dff3cbe6f0 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:33.473428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21cd1a-223e-419f-a3c7-f2dff3cbe6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:33.473428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af21cd1a-223e-419f-a3c7-f2dff3cbe6f0 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:33.474859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 573/1154] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:33.480198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eac42-c76b-41f7-a95b-1914f02a43e0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:33.480599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eac42-c76b-41f7-a95b-1914f02a43e0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:33.480770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eac42-c76b-41f7-a95b-1914f02a43e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:33.498433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-863eac42-c76b-41f7-a95b-1914f02a43e0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:33.500683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-863eac42-c76b-41f7-a95b-1914f02a43e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:33.568384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 582/1155] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:33.592422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21cd1a-223e-419f-a3c7-f2dff3cbe6f0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:33.674601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:33.699252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af21cd1a-223e-419f-a3c7-f2dff3cbe6f0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:33.733830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46806a7c-24b1-49c2-9f9c-ea3d83790110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:33.735205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46806a7c-24b1-49c2-9f9c-ea3d83790110 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:33.735490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46806a7c-24b1-49c2-9f9c-ea3d83790110 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:33.735775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46806a7c-24b1-49c2-9f9c-ea3d83790110 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:33.746340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46806a7c-24b1-49c2-9f9c-ea3d83790110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-46806a7c-24b1-49c2-9f9c-ea3d83790110 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:13:33.754504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:33.755656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:13:33.756673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 574/1156] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:13:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:33.762244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eac42-c76b-41f7-a95b-1914f02a43e0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:33.855300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:33.866893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eac42-c76b-41f7-a95b-1914f02a43e0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:33.900376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d8a31-52eb-4b1d-bb72-7af8f081a023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:33.901046 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d8a31-52eb-4b1d-bb72-7af8f081a023 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:33.901332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d8a31-52eb-4b1d-bb72-7af8f081a023 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:33.901610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d8a31-52eb-4b1d-bb72-7af8f081a023 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:33.902364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d8a31-52eb-4b1d-bb72-7af8f081a023 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:33.902713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d8a31-52eb-4b1d-bb72-7af8f081a023 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:33.904101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d8a31-52eb-4b1d-bb72-7af8f081a023 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:33.913331 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 583/1157] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:13:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:33.919496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46806a7c-24b1-49c2-9f9c-ea3d83790110 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:34.023449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46806a7c-24b1-49c2-9f9c-ea3d83790110 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:34.057445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90072a03-f83b-43d6-89dc-4b39b4a72bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:34.058470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90072a03-f83b-43d6-89dc-4b39b4a72bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:34.058773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90072a03-f83b-43d6-89dc-4b39b4a72bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:34.059048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90072a03-f83b-43d6-89dc-4b39b4a72bb6 None admin] PATH_INFO: `/v3/users/eee9195611f44c86b6e6d0b3483262c6` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:34.060237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90072a03-f83b-43d6-89dc-4b39b4a72bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=eee9195611f44c86b6e6d0b3483262c6)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:34.061665 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90072a03-f83b-43d6-89dc-4b39b4a72bb6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:34.063849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 575/1158] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:33 2022] GET /identity/v3/users/eee9195611f44c86b6e6d0b3483262c6 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:34.069692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d8a31-52eb-4b1d-bb72-7af8f081a023 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:34.171023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d8a31-52eb-4b1d-bb72-7af8f081a023 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:34.209319 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17ba6d-7386-4dc1-afb5-154cfdd73388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:34.209974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17ba6d-7386-4dc1-afb5-154cfdd73388 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:34.210262 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17ba6d-7386-4dc1-afb5-154cfdd73388 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:34.210543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17ba6d-7386-4dc1-afb5-154cfdd73388 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:34.231969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:34.282874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17ba6d-7386-4dc1-afb5-154cfdd73388 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:34.284282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17ba6d-7386-4dc1-afb5-154cfdd73388 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:34.396427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 584/1159] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:13:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:13:34.402379 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90072a03-f83b-43d6-89dc-4b39b4a72bb6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:34.412017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:34.508166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90072a03-f83b-43d6-89dc-4b39b4a72bb6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:34.541411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b2e22-67f4-41fa-bc57-b0fa306ccd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:34.542190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b2e22-67f4-41fa-bc57-b0fa306ccd71 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:34.542468 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b2e22-67f4-41fa-bc57-b0fa306ccd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:34.542771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b2e22-67f4-41fa-bc57-b0fa306ccd71 None admin] PATH_INFO: `/v3/projects/1c410de37660487783d03a96e667cd7d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:34.543945 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b2e22-67f4-41fa-bc57-b0fa306ccd71 None admin] RBAC: Authorizing `identity:get_project(project_id=1c410de37660487783d03a96e667cd7d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:34.545364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b2e22-67f4-41fa-bc57-b0fa306ccd71 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:34.548037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 576/1160] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:34 2022] GET /identity/v3/projects/1c410de37660487783d03a96e667cd7d => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:34.554724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17ba6d-7386-4dc1-afb5-154cfdd73388 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:34.608013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:34.657220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17ba6d-7386-4dc1-afb5-154cfdd73388 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:34.692647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee0697-ca95-4c13-90f3-4b80f18e497e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:34.693307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee0697-ca95-4c13-90f3-4b80f18e497e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:34.693587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee0697-ca95-4c13-90f3-4b80f18e497e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:34.693865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee0697-ca95-4c13-90f3-4b80f18e497e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:34.694736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee0697-ca95-4c13-90f3-4b80f18e497e None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=1c410de37660487783d03a96e667cd7d, user.id=eee9195611f44c86b6e6d0b3483262c6) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:34.695052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee0697-ca95-4c13-90f3-4b80f18e497e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:34.696521 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ee0697-ca95-4c13-90f3-4b80f18e497e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:34.703084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 585/1161] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:13:34 2022] GET /identity/v3/role_assignments?user.id=eee9195611f44c86b6e6d0b3483262c6&scope.project.id=1c410de37660487783d03a96e667cd7d&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:36.807762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d15714-62b8-4dce-b05f-d76a230957f7 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:36.808296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d15714-62b8-4dce-b05f-d76a230957f7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:36.808296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d15714-62b8-4dce-b05f-d76a230957f7 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:36.809672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 577/1162] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:36.815263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f86b4-b6b6-4f42-b51a-30c9b47c044d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:36.815689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f86b4-b6b6-4f42-b51a-30c9b47c044d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:36.815807 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f86b4-b6b6-4f42-b51a-30c9b47c044d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:36.832355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-825f86b4-b6b6-4f42-b51a-30c9b47c044d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:36.833545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-825f86b4-b6b6-4f42-b51a-30c9b47c044d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:36.900824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 586/1163] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:36.924855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d15714-62b8-4dce-b05f-d76a230957f7 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:37.003238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:37.033811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d15714-62b8-4dce-b05f-d76a230957f7 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:37.067787 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370822a0-9fe3-4d52-aba5-21b392a030b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:37.068360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370822a0-9fe3-4d52-aba5-21b392a030b6 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:37.068646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370822a0-9fe3-4d52-aba5-21b392a030b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:37.068926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370822a0-9fe3-4d52-aba5-21b392a030b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:37.136350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370822a0-9fe3-4d52-aba5-21b392a030b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:37.137805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370822a0-9fe3-4d52-aba5-21b392a030b6 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:37.211388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:37.236957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 578/1164] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:13:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:13:37.243356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f86b4-b6b6-4f42-b51a-30c9b47c044d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:37.288983 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:37.342171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f86b4-b6b6-4f42-b51a-30c9b47c044d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:37.379167 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11a5b9-3595-452d-8df9-941233ee3f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:37.379967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11a5b9-3595-452d-8df9-941233ee3f2f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:37.380248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11a5b9-3595-452d-8df9-941233ee3f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:37.380522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11a5b9-3595-452d-8df9-941233ee3f2f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:37.387254 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11a5b9-3595-452d-8df9-941233ee3f2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:37.388708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11a5b9-3595-452d-8df9-941233ee3f2f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-8f11a5b9-3595-452d-8df9-941233ee3f2f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:37.402976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:37.404228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:37.405272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 587/1165] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Jul 20 08:13:37 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:37.410450 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370822a0-9fe3-4d52-aba5-21b392a030b6 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:37.517689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370822a0-9fe3-4d52-aba5-21b392a030b6 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:37.552589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b798bcf-260f-4cc3-a37c-d5fe88008ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:37.553252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b798bcf-260f-4cc3-a37c-d5fe88008ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:37.553539 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b798bcf-260f-4cc3-a37c-d5fe88008ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:37.553814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b798bcf-260f-4cc3-a37c-d5fe88008ad9 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:37.554531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b798bcf-260f-4cc3-a37c-d5fe88008ad9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:37.554866 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b798bcf-260f-4cc3-a37c-d5fe88008ad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:37.556244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b798bcf-260f-4cc3-a37c-d5fe88008ad9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:37.564245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 579/1166] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Jul 20 08:13:37 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:39.614059 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e832a4-99bb-45c4-ad3f-96bc45c10548 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:39.614873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e832a4-99bb-45c4-ad3f-96bc45c10548 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:39.614873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e832a4-99bb-45c4-ad3f-96bc45c10548 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:39.616032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 588/1167] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:39.621538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dadd00-ddc8-417f-bf6d-0db05f9a984a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:39.621960 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dadd00-ddc8-417f-bf6d-0db05f9a984a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:39.622091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dadd00-ddc8-417f-bf6d-0db05f9a984a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:39.638569 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c4dadd00-ddc8-417f-bf6d-0db05f9a984a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:39.639791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c4dadd00-ddc8-417f-bf6d-0db05f9a984a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:39.671154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:39.710363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 580/1168] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:39.733832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e832a4-99bb-45c4-ad3f-96bc45c10548 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:39.838060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e832a4-99bb-45c4-ad3f-96bc45c10548 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:39.845483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:39.872886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd8cf3-7035-49d7-90b2-431ec29157c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:39.873461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd8cf3-7035-49d7-90b2-431ec29157c5 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:39.873745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd8cf3-7035-49d7-90b2-431ec29157c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:39.874019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd8cf3-7035-49d7-90b2-431ec29157c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:39.875309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd8cf3-7035-49d7-90b2-431ec29157c5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:39.876705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd8cf3-7035-49d7-90b2-431ec29157c5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:39.891753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 589/1169] 10.222.0.64 () {60 vars in 1138 bytes} [Wed Jul 20 08:13:39 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:13:42.000077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d8f3-f1be-484d-8787-ea46b3008eb7 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:42.000867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d8f3-f1be-484d-8787-ea46b3008eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:42.000867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d8f3-f1be-484d-8787-ea46b3008eb7 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:42.002135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 581/1170] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:42.007556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a9939-1b69-4653-8aa9-49973c1ebf4b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:42.007993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a9939-1b69-4653-8aa9-49973c1ebf4b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:42.008124 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a9939-1b69-4653-8aa9-49973c1ebf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:42.026506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d42a9939-1b69-4653-8aa9-49973c1ebf4b None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:42.027780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d42a9939-1b69-4653-8aa9-49973c1ebf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:42.096684 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 590/1171] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:42.120706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d8f3-f1be-484d-8787-ea46b3008eb7 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:42.225494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0261d8f3-f1be-484d-8787-ea46b3008eb7 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:42.260153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0a1e2-ba56-4d12-bfc5-f5cc6c52f27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:42.260939 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0a1e2-ba56-4d12-bfc5-f5cc6c52f27b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:42.261224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0a1e2-ba56-4d12-bfc5-f5cc6c52f27b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:42.261505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0a1e2-ba56-4d12-bfc5-f5cc6c52f27b None admin] PATH_INFO: `/v3/domains/bb14d99014cf426193ebef0e3815dbd7` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:42.272949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0a1e2-ba56-4d12-bfc5-f5cc6c52f27b None admin] RBAC: Authorizing `identity:get_domain(domain_id=bb14d99014cf426193ebef0e3815dbd7)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:42.274417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0a1e2-ba56-4d12-bfc5-f5cc6c52f27b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:42.276160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 582/1172] 10.222.0.64 () {56 vars in 1183 bytes} [Wed Jul 20 08:13:42 2022] GET /identity/v3/domains/bb14d99014cf426193ebef0e3815dbd7 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:42.282055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a9939-1b69-4653-8aa9-49973c1ebf4b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:42.377067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:42.383620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a9939-1b69-4653-8aa9-49973c1ebf4b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:42.418169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583128a3-cc76-4cb6-8e66-910ad577271b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:42.418865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583128a3-cc76-4cb6-8e66-910ad577271b None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:42.419150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583128a3-cc76-4cb6-8e66-910ad577271b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:42.419424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583128a3-cc76-4cb6-8e66-910ad577271b None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:42.420155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583128a3-cc76-4cb6-8e66-910ad577271b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:42.422128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583128a3-cc76-4cb6-8e66-910ad577271b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:42.448251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 591/1173] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:13:42 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:13:44.541557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752888-3af2-4fbc-81ea-b15d971c2002 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:44.542115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752888-3af2-4fbc-81ea-b15d971c2002 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:44.542115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94752888-3af2-4fbc-81ea-b15d971c2002 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:44.543512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 583/1174] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:44.548800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d39df-28fc-4465-ac62-38c5b5cb1742 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:44.549204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d39df-28fc-4465-ac62-38c5b5cb1742 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:44.549449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d39df-28fc-4465-ac62-38c5b5cb1742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:44.566973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b92d39df-28fc-4465-ac62-38c5b5cb1742 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:44.568171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b92d39df-28fc-4465-ac62-38c5b5cb1742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:44.645927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 592/1175] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:44.670298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94752888-3af2-4fbc-81ea-b15d971c2002 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:44.700938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:44.776016 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94752888-3af2-4fbc-81ea-b15d971c2002 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:44.809770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb433f-e7ac-47d8-817b-ed2c60e0369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:44.810667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb433f-e7ac-47d8-817b-ed2c60e0369b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:44.810958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb433f-e7ac-47d8-817b-ed2c60e0369b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:44.811238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb433f-e7ac-47d8-817b-ed2c60e0369b None admin] PATH_INFO: `/v3/domains/bb14d99014cf426193ebef0e3815dbd7` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:44.812985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb433f-e7ac-47d8-817b-ed2c60e0369b None admin] RBAC: Authorizing `identity:get_domain(domain_id=bb14d99014cf426193ebef0e3815dbd7)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:44.814427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb433f-e7ac-47d8-817b-ed2c60e0369b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:44.816086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 584/1176] 10.222.0.64 () {56 vars in 1183 bytes} [Wed Jul 20 08:13:44 2022] GET /identity/v3/domains/bb14d99014cf426193ebef0e3815dbd7 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:44.821904 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d39df-28fc-4465-ac62-38c5b5cb1742 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:44.905470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:44.929299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d39df-28fc-4465-ac62-38c5b5cb1742 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:44.966008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3e3d3-1018-4d33-8555-e26c0d11fa99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:44.966726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3e3d3-1018-4d33-8555-e26c0d11fa99 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:44.967013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3e3d3-1018-4d33-8555-e26c0d11fa99 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:44.967290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3e3d3-1018-4d33-8555-e26c0d11fa99 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:44.968041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3e3d3-1018-4d33-8555-e26c0d11fa99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:44.969591 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3e3d3-1018-4d33-8555-e26c0d11fa99 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:44.993222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 593/1177] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:13:44 2022] POST /identity/v3/users => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:13:47.110168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6690fcf-ce4f-47a7-9224-245b57c6e44b None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:47.110752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6690fcf-ce4f-47a7-9224-245b57c6e44b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:47.110752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6690fcf-ce4f-47a7-9224-245b57c6e44b None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:47.112108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 585/1178] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:47.117703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dffe1-45a8-4714-998f-f3d0616e93d7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:47.118119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dffe1-45a8-4714-998f-f3d0616e93d7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:47.118275 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dffe1-45a8-4714-998f-f3d0616e93d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:47.136753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-925dffe1-45a8-4714-998f-f3d0616e93d7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:47.137990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-925dffe1-45a8-4714-998f-f3d0616e93d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:47.209000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 594/1179] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:47.233447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6690fcf-ce4f-47a7-9224-245b57c6e44b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:47.328989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:47.336272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6690fcf-ce4f-47a7-9224-245b57c6e44b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:47.372760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2aff9-a377-4ebb-a778-ea0b77e3ff6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:47.373560 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2aff9-a377-4ebb-a778-ea0b77e3ff6a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:47.373839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2aff9-a377-4ebb-a778-ea0b77e3ff6a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:47.374114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2aff9-a377-4ebb-a778-ea0b77e3ff6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:47.384689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2aff9-a377-4ebb-a778-ea0b77e3ff6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-17e2aff9-a377-4ebb-a778-ea0b77e3ff6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:13:47.392442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:47.393546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:13:47.394536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 586/1180] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:13:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:47.399796 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dffe1-45a8-4714-998f-f3d0616e93d7 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:47.465195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:47.504229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dffe1-45a8-4714-998f-f3d0616e93d7 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:47.538043 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d23857-7815-4148-9c60-50d46896fd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:47.538734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d23857-7815-4148-9c60-50d46896fd9b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:47.539021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d23857-7815-4148-9c60-50d46896fd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:47.539300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d23857-7815-4148-9c60-50d46896fd9b None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:47.540048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d23857-7815-4148-9c60-50d46896fd9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:47.540360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d23857-7815-4148-9c60-50d46896fd9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:47.541747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d23857-7815-4148-9c60-50d46896fd9b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:47.550440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 595/1181] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:13:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:47.556299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2aff9-a377-4ebb-a778-ea0b77e3ff6a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:47.669112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2aff9-a377-4ebb-a778-ea0b77e3ff6a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:47.709224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcfafc-ffeb-4193-948a-0d01b1d032cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:47.710940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcfafc-ffeb-4193-948a-0d01b1d032cd None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:47.711227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcfafc-ffeb-4193-948a-0d01b1d032cd None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:47.711504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcfafc-ffeb-4193-948a-0d01b1d032cd None admin] PATH_INFO: `/v3/users/c8f399db1d0e4dd0b9465031b0001b43` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:47.728342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcfafc-ffeb-4193-948a-0d01b1d032cd None admin] RBAC: Authorizing `identity:get_user(user_id=c8f399db1d0e4dd0b9465031b0001b43)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:47.729806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcfafc-ffeb-4193-948a-0d01b1d032cd None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:47.731588 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 587/1182] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:47 2022] GET /identity/v3/users/c8f399db1d0e4dd0b9465031b0001b43 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:47.738220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d23857-7815-4148-9c60-50d46896fd9b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:47.840634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d23857-7815-4148-9c60-50d46896fd9b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:47.867348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:47.877077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52507b-dbfe-4241-856a-ab469bf717f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:47.877737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52507b-dbfe-4241-856a-ab469bf717f2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:47.878025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52507b-dbfe-4241-856a-ab469bf717f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:47.878305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52507b-dbfe-4241-856a-ab469bf717f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:47.955838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52507b-dbfe-4241-856a-ab469bf717f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:47.957227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52507b-dbfe-4241-856a-ab469bf717f2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:48.059226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 596/1183] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:13:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:13:48.065754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcfafc-ffeb-4193-948a-0d01b1d032cd None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:48.081549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:48.168512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcfafc-ffeb-4193-948a-0d01b1d032cd None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:48.203046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49205cf5-5e4f-472a-83f8-f52ef908123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:48.203910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49205cf5-5e4f-472a-83f8-f52ef908123a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:48.204195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49205cf5-5e4f-472a-83f8-f52ef908123a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:48.204477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49205cf5-5e4f-472a-83f8-f52ef908123a None admin] PATH_INFO: `/v3/projects/bd2fdb19e5bd4550815623afe2e5931e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:48.205699 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49205cf5-5e4f-472a-83f8-f52ef908123a None admin] RBAC: Authorizing `identity:get_project(project_id=bd2fdb19e5bd4550815623afe2e5931e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:48.207232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49205cf5-5e4f-472a-83f8-f52ef908123a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:48.208890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 588/1184] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:48 2022] GET /identity/v3/projects/bd2fdb19e5bd4550815623afe2e5931e => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:48.215083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52507b-dbfe-4241-856a-ab469bf717f2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:48.243204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:48.316714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52507b-dbfe-4241-856a-ab469bf717f2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:48.351088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32260c71-3bde-4e72-b638-307b6a1ebd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:48.351743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32260c71-3bde-4e72-b638-307b6a1ebd5c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:48.352029 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32260c71-3bde-4e72-b638-307b6a1ebd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:48.352304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32260c71-3bde-4e72-b638-307b6a1ebd5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:48.353138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32260c71-3bde-4e72-b638-307b6a1ebd5c None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=bd2fdb19e5bd4550815623afe2e5931e, user.id=c8f399db1d0e4dd0b9465031b0001b43) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:48.353446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32260c71-3bde-4e72-b638-307b6a1ebd5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:48.354920 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32260c71-3bde-4e72-b638-307b6a1ebd5c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:48.361223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 597/1185] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:13:48 2022] GET /identity/v3/role_assignments?user.id=c8f399db1d0e4dd0b9465031b0001b43&scope.project.id=bd2fdb19e5bd4550815623afe2e5931e&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:50.417066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f7bbb-fdc7-431a-9ef2-13f1c641c276 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:50.417596 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f7bbb-fdc7-431a-9ef2-13f1c641c276 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:50.417596 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f7bbb-fdc7-431a-9ef2-13f1c641c276 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:50.419042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 589/1186] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:50.424682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632f503-31fe-4c0a-a52b-051460b99d02 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:50.425125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632f503-31fe-4c0a-a52b-051460b99d02 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:50.425245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632f503-31fe-4c0a-a52b-051460b99d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:50.442450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4632f503-31fe-4c0a-a52b-051460b99d02 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:50.443691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4632f503-31fe-4c0a-a52b-051460b99d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:50.512716 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 598/1187] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:13:50.536348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f7bbb-fdc7-431a-9ef2-13f1c641c276 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:50.620586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:50.642965 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f7bbb-fdc7-431a-9ef2-13f1c641c276 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:50.681482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffc20d-bd2b-4429-a4fa-024f4f15edf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:50.682215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffc20d-bd2b-4429-a4fa-024f4f15edf9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:50.682495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffc20d-bd2b-4429-a4fa-024f4f15edf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:50.682798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffc20d-bd2b-4429-a4fa-024f4f15edf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:50.695598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ffc20d-bd2b-4429-a4fa-024f4f15edf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-67ffc20d-bd2b-4429-a4fa-024f4f15edf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:13:50.703626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:50.704863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:13:50.705893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 590/1188] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:13:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:50.711553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632f503-31fe-4c0a-a52b-051460b99d02 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:50.734440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:50.823927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632f503-31fe-4c0a-a52b-051460b99d02 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:50.861649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe3a92-9665-481b-a8e8-85d3d1854595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:50.862337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe3a92-9665-481b-a8e8-85d3d1854595 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:50.862654 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe3a92-9665-481b-a8e8-85d3d1854595 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:50.863204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe3a92-9665-481b-a8e8-85d3d1854595 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:50.863974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe3a92-9665-481b-a8e8-85d3d1854595 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:50.864287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe3a92-9665-481b-a8e8-85d3d1854595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:50.865696 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe3a92-9665-481b-a8e8-85d3d1854595 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:50.875819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 599/1189] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:13:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:50.881729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffc20d-bd2b-4429-a4fa-024f4f15edf9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:50.997119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ffc20d-bd2b-4429-a4fa-024f4f15edf9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:51.035038 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81372d94-a139-4f16-95c2-475fa9e53b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:51.035915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81372d94-a139-4f16-95c2-475fa9e53b2e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:51.036202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81372d94-a139-4f16-95c2-475fa9e53b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:51.036482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81372d94-a139-4f16-95c2-475fa9e53b2e None admin] PATH_INFO: `/v3/users/c8f399db1d0e4dd0b9465031b0001b43` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:51.037690 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81372d94-a139-4f16-95c2-475fa9e53b2e None admin] RBAC: Authorizing `identity:get_user(user_id=c8f399db1d0e4dd0b9465031b0001b43)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:51.039187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81372d94-a139-4f16-95c2-475fa9e53b2e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:51.040831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 591/1190] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:50 2022] GET /identity/v3/users/c8f399db1d0e4dd0b9465031b0001b43 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:51.047368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe3a92-9665-481b-a8e8-85d3d1854595 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:51.137477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:51.158482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe3a92-9665-481b-a8e8-85d3d1854595 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:51.195772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98c69e-b513-4435-9eba-1fec014da2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:51.196638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98c69e-b513-4435-9eba-1fec014da2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:51.196917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98c69e-b513-4435-9eba-1fec014da2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:51.197189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98c69e-b513-4435-9eba-1fec014da2a1 None admin] PATH_INFO: `/v3/projects/bd2fdb19e5bd4550815623afe2e5931e` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:51.198377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98c69e-b513-4435-9eba-1fec014da2a1 None admin] RBAC: Authorizing `identity:get_project(project_id=bd2fdb19e5bd4550815623afe2e5931e)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:51.199858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98c69e-b513-4435-9eba-1fec014da2a1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:51.201575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 600/1191] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:51 2022] GET /identity/v3/projects/bd2fdb19e5bd4550815623afe2e5931e => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:51.208577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81372d94-a139-4f16-95c2-475fa9e53b2e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:51.221976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:51.326450 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81372d94-a139-4f16-95c2-475fa9e53b2e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:51.367833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b640a6-8b8c-468c-8544-7379e061fb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:51.368724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b640a6-8b8c-468c-8544-7379e061fb4d None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:51.369009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b640a6-8b8c-468c-8544-7379e061fb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:51.369289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b640a6-8b8c-468c-8544-7379e061fb4d None admin] PATH_INFO: `/v3/projects/bd2fdb19e5bd4550815623afe2e5931e/users/c8f399db1d0e4dd0b9465031b0001b43/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:51.371294 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b640a6-8b8c-468c-8544-7379e061fb4d None admin] RBAC: Authorizing `identity:create_grant(project_id=bd2fdb19e5bd4550815623afe2e5931e, user_id=c8f399db1d0e4dd0b9465031b0001b43, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:51.372739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b640a6-8b8c-468c-8544-7379e061fb4d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:51.388070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 592/1192] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:13:51 2022] PUT /identity/v3/projects/bd2fdb19e5bd4550815623afe2e5931e/users/c8f399db1d0e4dd0b9465031b0001b43/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:13:53.449599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf4079-5c10-431f-acf8-dc85b3b8b307 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:53.450172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf4079-5c10-431f-acf8-dc85b3b8b307 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:53.450172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf4079-5c10-431f-acf8-dc85b3b8b307 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:53.451548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 601/1193] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:13:53.457068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ce8e9-bd2d-42c4-9977-f891b7169dac None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:53.457561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ce8e9-bd2d-42c4-9977-f891b7169dac None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:53.457675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ce8e9-bd2d-42c4-9977-f891b7169dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:53.476770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ae3ce8e9-bd2d-42c4-9977-f891b7169dac None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:13:53.477980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ae3ce8e9-bd2d-42c4-9977-f891b7169dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:13:53.548633 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:53.558656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 593/1194] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:13:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:13:53.584703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf4079-5c10-431f-acf8-dc85b3b8b307 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:53.697248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf4079-5c10-431f-acf8-dc85b3b8b307 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:53.733389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae98be3-984d-42f2-8e34-a7f2a06cf735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:53.734149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae98be3-984d-42f2-8e34-a7f2a06cf735 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:53.734432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae98be3-984d-42f2-8e34-a7f2a06cf735 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:53.734741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae98be3-984d-42f2-8e34-a7f2a06cf735 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:53.745332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae98be3-984d-42f2-8e34-a7f2a06cf735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-cae98be3-984d-42f2-8e34-a7f2a06cf735 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:13:53.753272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:13:53.754448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:13:53.755638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 602/1195] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:13:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:13:53.760782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ce8e9-bd2d-42c4-9977-f891b7169dac None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:53.870727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ce8e9-bd2d-42c4-9977-f891b7169dac None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:53.905427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3d88f-d5a7-4a2f-a752-5f8a25ea705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:53.906203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3d88f-d5a7-4a2f-a752-5f8a25ea705e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:53.907024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3d88f-d5a7-4a2f-a752-5f8a25ea705e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:53.907302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3d88f-d5a7-4a2f-a752-5f8a25ea705e None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:53.908064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3d88f-d5a7-4a2f-a752-5f8a25ea705e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:53.908374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3d88f-d5a7-4a2f-a752-5f8a25ea705e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:53.909775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3d88f-d5a7-4a2f-a752-5f8a25ea705e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:53.918724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 594/1196] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:13:53.924664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae98be3-984d-42f2-8e34-a7f2a06cf735 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:53.937995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:54.029019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae98be3-984d-42f2-8e34-a7f2a06cf735 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:54.064542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ce35c-a26b-4530-a4c6-55f7a8e4ca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:54.065418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce35c-a26b-4530-a4c6-55f7a8e4ca9e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:54.065701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce35c-a26b-4530-a4c6-55f7a8e4ca9e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:54.065979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce35c-a26b-4530-a4c6-55f7a8e4ca9e None admin] PATH_INFO: `/v3/users/c8f399db1d0e4dd0b9465031b0001b43` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:54.067207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ce35c-a26b-4530-a4c6-55f7a8e4ca9e None admin] RBAC: Authorizing `identity:get_user(user_id=c8f399db1d0e4dd0b9465031b0001b43)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:54.068662 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ce35c-a26b-4530-a4c6-55f7a8e4ca9e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:54.070308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 603/1197] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Jul 20 08:13:53 2022] GET /identity/v3/users/c8f399db1d0e4dd0b9465031b0001b43 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:54.076231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3d88f-d5a7-4a2f-a752-5f8a25ea705e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:54.139621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:54.179733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3d88f-d5a7-4a2f-a752-5f8a25ea705e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:54.215460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e89ba-2ae9-4268-a30d-580df3dbad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:54.216112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e89ba-2ae9-4268-a30d-580df3dbad9d None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:54.216412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e89ba-2ae9-4268-a30d-580df3dbad9d None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:54.216687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e89ba-2ae9-4268-a30d-580df3dbad9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:54.288647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e89ba-2ae9-4268-a30d-580df3dbad9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:54.290049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e89ba-2ae9-4268-a30d-580df3dbad9d None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:54.352127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:54.394393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 595/1198] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:13:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:13:54.400506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ce35c-a26b-4530-a4c6-55f7a8e4ca9e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:54.482876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:13:54.507914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ce35c-a26b-4530-a4c6-55f7a8e4ca9e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:54.543245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29322ab7-1e7b-4eb0-baf8-554024fbdbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:54.544105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29322ab7-1e7b-4eb0-baf8-554024fbdbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:54.544385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29322ab7-1e7b-4eb0-baf8-554024fbdbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:54.544659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29322ab7-1e7b-4eb0-baf8-554024fbdbe6 None admin] PATH_INFO: `/v3/projects/bd2fdb19e5bd4550815623afe2e5931e` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:54.545841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29322ab7-1e7b-4eb0-baf8-554024fbdbe6 None admin] RBAC: Authorizing `identity:get_project(project_id=bd2fdb19e5bd4550815623afe2e5931e)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:54.548510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29322ab7-1e7b-4eb0-baf8-554024fbdbe6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:54.550238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 604/1199] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Jul 20 08:13:54 2022] GET /identity/v3/projects/bd2fdb19e5bd4550815623afe2e5931e => generated 361 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:13:54.556732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e89ba-2ae9-4268-a30d-580df3dbad9d None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:13:54.663453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e89ba-2ae9-4268-a30d-580df3dbad9d None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:13:54.704320 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee395cb9-a160-416c-9739-9193cec8744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:13:54.704974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee395cb9-a160-416c-9739-9193cec8744e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:13:54.705257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee395cb9-a160-416c-9739-9193cec8744e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:13:54.705531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee395cb9-a160-416c-9739-9193cec8744e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:13:54.706365 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee395cb9-a160-416c-9739-9193cec8744e None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=bd2fdb19e5bd4550815623afe2e5931e, user.id=c8f399db1d0e4dd0b9465031b0001b43) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:13:54.706701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee395cb9-a160-416c-9739-9193cec8744e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:13:54.708135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee395cb9-a160-416c-9739-9193cec8744e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:13:54.714241 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 596/1200] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:13:54 2022] GET /identity/v3/role_assignments?user.id=c8f399db1d0e4dd0b9465031b0001b43&scope.project.id=bd2fdb19e5bd4550815623afe2e5931e&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:14:04.538796 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=27 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:14:04.711695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=87 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:14:59.189912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294abcd-8ab1-41d1-b4da-aa50084ae363 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:14:59.190564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294abcd-8ab1-41d1-b4da-aa50084ae363 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:14:59.190564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294abcd-8ab1-41d1-b4da-aa50084ae363 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:14:59.191902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 605/1201] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:14:59.198167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b88bc-eb7c-45da-97f3-d5c6d7a8339d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:14:59.200193 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b88bc-eb7c-45da-97f3-d5c6d7a8339d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:14:59.200193 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b88bc-eb7c-45da-97f3-d5c6d7a8339d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:14:59.221872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ec7b88bc-eb7c-45da-97f3-d5c6d7a8339d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:14:59.223326 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ec7b88bc-eb7c-45da-97f3-d5c6d7a8339d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:14:59.321418 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 597/1202] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:14:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:14:59.357337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294abcd-8ab1-41d1-b4da-aa50084ae363 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:14:59.546338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294abcd-8ab1-41d1-b4da-aa50084ae363 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:14:59.612550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d7254-a407-4088-abdf-986366b86d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:14:59.613452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d7254-a407-4088-abdf-986366b86d0c None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:14:59.613871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d7254-a407-4088-abdf-986366b86d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:14:59.614353 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1d7254-a407-4088-abdf-986366b86d0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:14:59.631844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d7254-a407-4088-abdf-986366b86d0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:14:59.633496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1d7254-a407-4088-abdf-986366b86d0c None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-5a1d7254-a407-4088-abdf-986366b86d0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:14:59.659693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:14:59.661104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:14:59.662086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:14:59.662086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:14:59.662086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:14:59.662086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:14:59.662086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 606/1203] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:14:59.669065 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b88bc-eb7c-45da-97f3-d5c6d7a8339d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:14:59.840306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b88bc-eb7c-45da-97f3-d5c6d7a8339d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:14:59.878304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:14:59.886307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fd1a6-8659-4852-b075-5172228d01ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:14:59.887199 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fd1a6-8659-4852-b075-5172228d01ac None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:14:59.887610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fd1a6-8659-4852-b075-5172228d01ac None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:14:59.888265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fd1a6-8659-4852-b075-5172228d01ac None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:14:59.889196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fd1a6-8659-4852-b075-5172228d01ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:14:59.889512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fd1a6-8659-4852-b075-5172228d01ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:14:59.891699 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fd1a6-8659-4852-b075-5172228d01ac None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:14:59.907910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 598/1204] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:14:59.915681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d7254-a407-4088-abdf-986366b86d0c None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:00.003799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:00.087914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1d7254-a407-4088-abdf-986366b86d0c None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:00.139143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5234e2-29c5-4ee9-b78c-855214294cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:00.139726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5234e2-29c5-4ee9-b78c-855214294cf1 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:00.140094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5234e2-29c5-4ee9-b78c-855214294cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:00.140328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5234e2-29c5-4ee9-b78c-855214294cf1 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:00.141116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5234e2-29c5-4ee9-b78c-855214294cf1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:00.142761 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5234e2-29c5-4ee9-b78c-855214294cf1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:00.169264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 607/1205] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Jul 20 08:14:59 2022] POST /identity/v3/users => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:15:02.696394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082d35f-9b52-4ba8-a990-92fbb3c9c071 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:02.696969 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082d35f-9b52-4ba8-a990-92fbb3c9c071 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:02.697255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082d35f-9b52-4ba8-a990-92fbb3c9c071 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:02.700402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 599/1206] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:15:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:15:02.706754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34b737-be32-4695-ac0c-efd1c695686d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:02.707466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34b737-be32-4695-ac0c-efd1c695686d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:02.707466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34b737-be32-4695-ac0c-efd1c695686d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:02.725888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8e34b737-be32-4695-ac0c-efd1c695686d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:15:02.727171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8e34b737-be32-4695-ac0c-efd1c695686d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:15:02.837826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 608/1207] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:15:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:15:02.867667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082d35f-9b52-4ba8-a990-92fbb3c9c071 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:03.023737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082d35f-9b52-4ba8-a990-92fbb3c9c071 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:03.074684 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee4566-fce9-4613-a0ae-59db33e02f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:03.075530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee4566-fce9-4613-a0ae-59db33e02f69 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:03.075817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee4566-fce9-4613-a0ae-59db33e02f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:03.076109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee4566-fce9-4613-a0ae-59db33e02f69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:03.090204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee4566-fce9-4613-a0ae-59db33e02f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-86ee4566-fce9-4613-a0ae-59db33e02f69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:15:03.099850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:03.107110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:03.109595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 600/1208] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:15:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:03.115167 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34b737-be32-4695-ac0c-efd1c695686d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:03.167887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:03.247596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34b737-be32-4695-ac0c-efd1c695686d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:03.300373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971be571-cded-40c7-894d-ec8be013b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:03.301159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971be571-cded-40c7-894d-ec8be013b90f None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:03.301488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971be571-cded-40c7-894d-ec8be013b90f None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:03.301729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971be571-cded-40c7-894d-ec8be013b90f None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:03.302557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971be571-cded-40c7-894d-ec8be013b90f None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:03.302986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971be571-cded-40c7-894d-ec8be013b90f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:03.304597 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971be571-cded-40c7-894d-ec8be013b90f None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:03.320776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 609/1209] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:15:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:15:03.329320 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee4566-fce9-4613-a0ae-59db33e02f69 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:03.399156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:03.463733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee4566-fce9-4613-a0ae-59db33e02f69 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:03.507371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d0891-36ad-4925-b095-1d803267c037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:03.508050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d0891-36ad-4925-b095-1d803267c037 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:03.508304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d0891-36ad-4925-b095-1d803267c037 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:03.508654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d0891-36ad-4925-b095-1d803267c037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:03.523905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d0891-36ad-4925-b095-1d803267c037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:03.525499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d0891-36ad-4925-b095-1d803267c037 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b20d0891-36ad-4925-b095-1d803267c037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:03.551432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:03.552911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:03.554116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:03.554116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:03.554116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:03.554116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:03.554116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 601/1210] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:15:03.560114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971be571-cded-40c7-894d-ec8be013b90f None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:03.716578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:03.720601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971be571-cded-40c7-894d-ec8be013b90f None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:03.769442 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbeab61-021b-49f2-9bf0-9d28e7683c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:03.770233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbeab61-021b-49f2-9bf0-9d28e7683c17 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:03.770637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbeab61-021b-49f2-9bf0-9d28e7683c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:03.770934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbeab61-021b-49f2-9bf0-9d28e7683c17 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:03.771963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbeab61-021b-49f2-9bf0-9d28e7683c17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:03.772346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbeab61-021b-49f2-9bf0-9d28e7683c17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:03.774234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbeab61-021b-49f2-9bf0-9d28e7683c17 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:03.797950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 610/1211] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:03.799431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d0891-36ad-4925-b095-1d803267c037 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:03.946166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d0891-36ad-4925-b095-1d803267c037 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:03.948048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:03.992270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8ad57-c100-4a76-8fc6-03be6678cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:03.993153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8ad57-c100-4a76-8fc6-03be6678cb45 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:03.993498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8ad57-c100-4a76-8fc6-03be6678cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:03.993759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8ad57-c100-4a76-8fc6-03be6678cb45 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:04.009885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8ad57-c100-4a76-8fc6-03be6678cb45 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:04.011570 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8ad57-c100-4a76-8fc6-03be6678cb45 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-25b8ad57-c100-4a76-8fc6-03be6678cb45 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:15:04.039095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:04.040121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:04.041055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:04.041055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:04.041055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:15:04.041055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:15:04.041055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:15:04.041055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:15:04.041055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:15:04.041055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:15:04.041055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:04.041055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:04.047954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbeab61-021b-49f2-9bf0-9d28e7683c17 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:04.048462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 602/1212] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:04.226528 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbeab61-021b-49f2-9bf0-9d28e7683c17 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:04.274018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd898561-9010-4c25-994c-e88608d16413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:04.275105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898561-9010-4c25-994c-e88608d16413 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:04.275444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898561-9010-4c25-994c-e88608d16413 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:04.275713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd898561-9010-4c25-994c-e88608d16413 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:04.276562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd898561-9010-4c25-994c-e88608d16413 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:04.276862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd898561-9010-4c25-994c-e88608d16413 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:04.278420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd898561-9010-4c25-994c-e88608d16413 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:04.322439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 611/1213] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Jul 20 08:15:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:04.325139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8ad57-c100-4a76-8fc6-03be6678cb45 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:04.516026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8ad57-c100-4a76-8fc6-03be6678cb45 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:04.561595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5e3af-68a5-4d45-a9df-fcb71f5be4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:04.562969 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e3af-68a5-4d45-a9df-fcb71f5be4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:04.563167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e3af-68a5-4d45-a9df-fcb71f5be4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:04.563512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5e3af-68a5-4d45-a9df-fcb71f5be4b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:04.606504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:04.660310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5e3af-68a5-4d45-a9df-fcb71f5be4b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:04.662109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5e3af-68a5-4d45-a9df-fcb71f5be4b7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:04.832174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 603/1214] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:15:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:15:04.838346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd898561-9010-4c25-994c-e88608d16413 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:04.862774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:04.984917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd898561-9010-4c25-994c-e88608d16413 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:05.049468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d071d-884f-4f8c-ad5c-5ee2a696ce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:05.051205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d071d-884f-4f8c-ad5c-5ee2a696ce63 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:05.051649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d071d-884f-4f8c-ad5c-5ee2a696ce63 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:05.052133 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d071d-884f-4f8c-ad5c-5ee2a696ce63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:05.069091 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d071d-884f-4f8c-ad5c-5ee2a696ce63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:05.071674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d071d-884f-4f8c-ad5c-5ee2a696ce63 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-065d071d-884f-4f8c-ad5c-5ee2a696ce63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:05.097156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:05.098168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:05.099467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:05.099467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:05.099467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:05.099467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:05.100150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 612/1215] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:05.107523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5e3af-68a5-4d45-a9df-fcb71f5be4b7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:05.209344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:05.260269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5e3af-68a5-4d45-a9df-fcb71f5be4b7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:05.308844 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e131297-90fc-464b-ae41-94a86702376e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:05.310042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e131297-90fc-464b-ae41-94a86702376e None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:05.310552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e131297-90fc-464b-ae41-94a86702376e None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:05.311082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e131297-90fc-464b-ae41-94a86702376e None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:05.313164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e131297-90fc-464b-ae41-94a86702376e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:05.313741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e131297-90fc-464b-ae41-94a86702376e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:05.316209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e131297-90fc-464b-ae41-94a86702376e None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:05.339547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 604/1216] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:05.345882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d071d-884f-4f8c-ad5c-5ee2a696ce63 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:05.466373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:05.512341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d071d-884f-4f8c-ad5c-5ee2a696ce63 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:05.567739 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90559dfe-e9cf-4e70-b06d-6f48f79a40f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:05.568969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90559dfe-e9cf-4e70-b06d-6f48f79a40f2 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:05.569422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90559dfe-e9cf-4e70-b06d-6f48f79a40f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:05.569843 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90559dfe-e9cf-4e70-b06d-6f48f79a40f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:05.580628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90559dfe-e9cf-4e70-b06d-6f48f79a40f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:05.582565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90559dfe-e9cf-4e70-b06d-6f48f79a40f2 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-90559dfe-e9cf-4e70-b06d-6f48f79a40f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:15:05.599964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:05.601260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:05.602312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:15:05.602312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:15:05.602312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:15:05.602312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:15:05.602312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:15:05.602312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:15:05.602312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:05.602312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:05.602312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 613/1217] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:15:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:05.607812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e131297-90fc-464b-ae41-94a86702376e None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:05.746919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e131297-90fc-464b-ae41-94a86702376e None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:05.776074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:05.791103 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5064403-8bbf-4b7c-9117-0fe435e85bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:05.791841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5064403-8bbf-4b7c-9117-0fe435e85bcc None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:05.792182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5064403-8bbf-4b7c-9117-0fe435e85bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:05.792593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5064403-8bbf-4b7c-9117-0fe435e85bcc None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:05.793463 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5064403-8bbf-4b7c-9117-0fe435e85bcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:05.793850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5064403-8bbf-4b7c-9117-0fe435e85bcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:05.795656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5064403-8bbf-4b7c-9117-0fe435e85bcc None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:05.820869 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 605/1218] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:15:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:05.823674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90559dfe-e9cf-4e70-b06d-6f48f79a40f2 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:05.986897 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90559dfe-e9cf-4e70-b06d-6f48f79a40f2 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:06.040629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:06.051930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f2443-6ef8-4f6c-927a-220ea106315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:06.054611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f2443-6ef8-4f6c-927a-220ea106315a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:06.054611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f2443-6ef8-4f6c-927a-220ea106315a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:06.054611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f2443-6ef8-4f6c-927a-220ea106315a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:06.054611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f2443-6ef8-4f6c-927a-220ea106315a None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=928054a577d14522bd0f00417f4f8c1f) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:06.054611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f2443-6ef8-4f6c-927a-220ea106315a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:06.056346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f2443-6ef8-4f6c-927a-220ea106315a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:06.063560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 614/1219] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:15:05 2022] GET /identity/v3/role_assignments?user.id=928054a577d14522bd0f00417f4f8c1f&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:08.409908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d038baf-76cf-44f2-8e77-0d87cd489abb None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:08.410708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d038baf-76cf-44f2-8e77-0d87cd489abb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:08.410708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d038baf-76cf-44f2-8e77-0d87cd489abb None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:08.412247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 606/1220] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:15:08.419889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a19f97-fe35-47ef-b81f-6bceae6d9553 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:08.420463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a19f97-fe35-47ef-b81f-6bceae6d9553 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:08.420594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a19f97-fe35-47ef-b81f-6bceae6d9553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:08.448603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-35a19f97-fe35-47ef-b81f-6bceae6d9553 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:15:08.450480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-35a19f97-fe35-47ef-b81f-6bceae6d9553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:15:08.547690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 615/1221] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:15:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 08:15:08.577848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d038baf-76cf-44f2-8e77-0d87cd489abb None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:08.722705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d038baf-76cf-44f2-8e77-0d87cd489abb None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:08.772282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582a517-c753-4a4c-a7bb-bc4127cd4998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:08.773434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582a517-c753-4a4c-a7bb-bc4127cd4998 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:08.773784 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582a517-c753-4a4c-a7bb-bc4127cd4998 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:08.774124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582a517-c753-4a4c-a7bb-bc4127cd4998 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:08.791990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582a517-c753-4a4c-a7bb-bc4127cd4998 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e582a517-c753-4a4c-a7bb-bc4127cd4998 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:15:08.806489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:08.808916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:08.809759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 607/1222] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:15:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:08.818485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a19f97-fe35-47ef-b81f-6bceae6d9553 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:08.967348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a19f97-fe35-47ef-b81f-6bceae6d9553 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:09.008424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:09.023479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2217e2d-6d84-4f29-99f8-609155d34c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:09.024529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2217e2d-6d84-4f29-99f8-609155d34c09 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:09.025033 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2217e2d-6d84-4f29-99f8-609155d34c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:09.025647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2217e2d-6d84-4f29-99f8-609155d34c09 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:09.026988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2217e2d-6d84-4f29-99f8-609155d34c09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:09.027669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2217e2d-6d84-4f29-99f8-609155d34c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:09.029423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2217e2d-6d84-4f29-99f8-609155d34c09 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:09.039793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 616/1223] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:15:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:09.046933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582a517-c753-4a4c-a7bb-bc4127cd4998 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:09.106642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:09.178428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582a517-c753-4a4c-a7bb-bc4127cd4998 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:09.228760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516376d-84e9-4697-892b-2bfb1d384f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:09.229574 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516376d-84e9-4697-892b-2bfb1d384f5f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:09.229953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516376d-84e9-4697-892b-2bfb1d384f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:09.230290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f516376d-84e9-4697-892b-2bfb1d384f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:09.246380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516376d-84e9-4697-892b-2bfb1d384f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:09.248347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f516376d-84e9-4697-892b-2bfb1d384f5f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-f516376d-84e9-4697-892b-2bfb1d384f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:09.271493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:09.273125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:09.275035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:09.275035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:09.275035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:09.275035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:09.275035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 608/1224] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:09.279338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2217e2d-6d84-4f29-99f8-609155d34c09 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:09.427563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2217e2d-6d84-4f29-99f8-609155d34c09 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:09.487868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95806694-6ac9-4eb1-a879-a9705441c482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:09.488744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95806694-6ac9-4eb1-a879-a9705441c482 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:09.489233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95806694-6ac9-4eb1-a879-a9705441c482 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:09.489748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95806694-6ac9-4eb1-a879-a9705441c482 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:09.491195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95806694-6ac9-4eb1-a879-a9705441c482 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:09.491654 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95806694-6ac9-4eb1-a879-a9705441c482 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:09.493278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95806694-6ac9-4eb1-a879-a9705441c482 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:09.517041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 617/1225] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:09.525781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516376d-84e9-4697-892b-2bfb1d384f5f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:09.695067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:09.708774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f516376d-84e9-4697-892b-2bfb1d384f5f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:09.758231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceaf628-95c8-4e67-9813-fad7fb41569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:09.759290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceaf628-95c8-4e67-9813-fad7fb41569c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:09.759612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceaf628-95c8-4e67-9813-fad7fb41569c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:09.759913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceaf628-95c8-4e67-9813-fad7fb41569c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:09.770471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceaf628-95c8-4e67-9813-fad7fb41569c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:09.772389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceaf628-95c8-4e67-9813-fad7fb41569c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-9ceaf628-95c8-4e67-9813-fad7fb41569c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:15:09.791371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:09.792830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:09.794647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:09.794647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:09.794647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:15:09.794647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:15:09.794647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:15:09.794647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:15:09.794647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:15:09.794647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:15:09.794647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:09.794647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:09.799032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 609/1226] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:09.800391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95806694-6ac9-4eb1-a879-a9705441c482 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:09.867427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:09.965052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95806694-6ac9-4eb1-a879-a9705441c482 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:10.045681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294f47a-5813-4680-8446-b476c868d8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:10.047437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294f47a-5813-4680-8446-b476c868d8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:10.048005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294f47a-5813-4680-8446-b476c868d8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:10.048652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294f47a-5813-4680-8446-b476c868d8d9 None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:10.050970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294f47a-5813-4680-8446-b476c868d8d9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:10.051622 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294f47a-5813-4680-8446-b476c868d8d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:10.054786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294f47a-5813-4680-8446-b476c868d8d9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:10.083348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 618/1227] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Jul 20 08:15:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:15:10.094878 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceaf628-95c8-4e67-9813-fad7fb41569c None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:10.257874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceaf628-95c8-4e67-9813-fad7fb41569c None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:10.309066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1940944-8e34-4772-a04d-5dfa8f24a275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:10.310186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1940944-8e34-4772-a04d-5dfa8f24a275 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:10.310659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1940944-8e34-4772-a04d-5dfa8f24a275 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:10.311188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1940944-8e34-4772-a04d-5dfa8f24a275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:10.331339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1940944-8e34-4772-a04d-5dfa8f24a275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:10.333206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1940944-8e34-4772-a04d-5dfa8f24a275 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-e1940944-8e34-4772-a04d-5dfa8f24a275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:10.357178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:10.359333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:10.361575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:10.361575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:10.361575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:10.361575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:10.361575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 610/1228] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:15:10.368944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294f47a-5813-4680-8446-b476c868d8d9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:10.529690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294f47a-5813-4680-8446-b476c868d8d9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:10.537495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:10.583256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711fb78-1a53-48e6-8df6-bbbbbeb3b03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:10.583840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711fb78-1a53-48e6-8df6-bbbbbeb3b03b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:10.584165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711fb78-1a53-48e6-8df6-bbbbbeb3b03b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:10.584390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711fb78-1a53-48e6-8df6-bbbbbeb3b03b None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:10.585344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711fb78-1a53-48e6-8df6-bbbbbeb3b03b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:10.585645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711fb78-1a53-48e6-8df6-bbbbbeb3b03b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:10.587285 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711fb78-1a53-48e6-8df6-bbbbbeb3b03b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:10.603099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 619/1229] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:10.610009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1940944-8e34-4772-a04d-5dfa8f24a275 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:10.628024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:10.788732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1940944-8e34-4772-a04d-5dfa8f24a275 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:10.847730 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d993f-a51d-4bb6-a119-aa4b44751f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:10.848624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d993f-a51d-4bb6-a119-aa4b44751f31 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:10.849052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d993f-a51d-4bb6-a119-aa4b44751f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:10.849496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d993f-a51d-4bb6-a119-aa4b44751f31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:10.861466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d993f-a51d-4bb6-a119-aa4b44751f31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:10.863469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d993f-a51d-4bb6-a119-aa4b44751f31 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-978d993f-a51d-4bb6-a119-aa4b44751f31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:15:10.880996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:10.883213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:10.885120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:15:10.885120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:15:10.885120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:15:10.885120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:15:10.885120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:15:10.885120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:15:10.885120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:10.885120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:10.885120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 611/1230] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:10.888994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711fb78-1a53-48e6-8df6-bbbbbeb3b03b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:11.065682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711fb78-1a53-48e6-8df6-bbbbbeb3b03b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:11.127317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9a8ac-4c2b-4672-808c-e95b0f87bcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:11.128068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9a8ac-4c2b-4672-808c-e95b0f87bcf6 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:11.128421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9a8ac-4c2b-4672-808c-e95b0f87bcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:11.128674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9a8ac-4c2b-4672-808c-e95b0f87bcf6 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:11.129573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9a8ac-4c2b-4672-808c-e95b0f87bcf6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:11.129883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9a8ac-4c2b-4672-808c-e95b0f87bcf6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:11.131379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d9a8ac-4c2b-4672-808c-e95b0f87bcf6 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:11.152752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 620/1231] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:15:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:11.160996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d993f-a51d-4bb6-a119-aa4b44751f31 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:11.255784 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:11.312682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d993f-a51d-4bb6-a119-aa4b44751f31 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:11.360938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d974970c-7069-4d4e-8092-8b7d2a2bdf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:11.362029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974970c-7069-4d4e-8092-8b7d2a2bdf94 None admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:11.362396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974970c-7069-4d4e-8092-8b7d2a2bdf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:11.362739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d974970c-7069-4d4e-8092-8b7d2a2bdf94 None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/928054a577d14522bd0f00417f4f8c1f/roles/d60298c676854e83b88df984e42773da` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:11.388227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974970c-7069-4d4e-8092-8b7d2a2bdf94 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=928054a577d14522bd0f00417f4f8c1f, role_id=d60298c676854e83b88df984e42773da)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:11.389915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d974970c-7069-4d4e-8092-8b7d2a2bdf94 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:11.412288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 612/1232] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:15:11 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/928054a577d14522bd0f00417f4f8c1f/roles/d60298c676854e83b88df984e42773da => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:15:13.773480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf55c95-ce0b-4abd-9ca8-36cbaeebca52 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:13.774034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf55c95-ce0b-4abd-9ca8-36cbaeebca52 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:13.774493 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf55c95-ce0b-4abd-9ca8-36cbaeebca52 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:13.776901 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 621/1233] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:15:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:15:13.784769 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eb4fe-f24f-4eaf-9866-aa84031cf6c0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:13.785362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eb4fe-f24f-4eaf-9866-aa84031cf6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:13.785741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2eb4fe-f24f-4eaf-9866-aa84031cf6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:13.810798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0c2eb4fe-f24f-4eaf-9866-aa84031cf6c0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:15:13.812707 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0c2eb4fe-f24f-4eaf-9866-aa84031cf6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:15:13.909388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 613/1234] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:15:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:15:13.934560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf55c95-ce0b-4abd-9ca8-36cbaeebca52 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:13.969229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:14.077408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf55c95-ce0b-4abd-9ca8-36cbaeebca52 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:14.134119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b82c3-42b2-48af-800a-4dc42f39c18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:14.135032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b82c3-42b2-48af-800a-4dc42f39c18d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:14.135358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b82c3-42b2-48af-800a-4dc42f39c18d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:14.135652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b82c3-42b2-48af-800a-4dc42f39c18d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:14.154827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b82c3-42b2-48af-800a-4dc42f39c18d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0c9b82c3-42b2-48af-800a-4dc42f39c18d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:15:14.169523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:14.172302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 08:15:14.174032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:14.178839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 622/1235] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Jul 20 08:15:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:14.181808 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eb4fe-f24f-4eaf-9866-aa84031cf6c0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:14.268885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:14.348051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2eb4fe-f24f-4eaf-9866-aa84031cf6c0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:14.408741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209b9a4-a578-49f5-86b6-c72a2cfbcb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:14.409696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209b9a4-a578-49f5-86b6-c72a2cfbcb81 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:14.410196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209b9a4-a578-49f5-86b6-c72a2cfbcb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:14.410797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209b9a4-a578-49f5-86b6-c72a2cfbcb81 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:14.412704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209b9a4-a578-49f5-86b6-c72a2cfbcb81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:14.413428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209b9a4-a578-49f5-86b6-c72a2cfbcb81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:14.416377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209b9a4-a578-49f5-86b6-c72a2cfbcb81 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:14.432188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 614/1236] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Jul 20 08:15:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:14.441781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b82c3-42b2-48af-800a-4dc42f39c18d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:14.570779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:14.595316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b82c3-42b2-48af-800a-4dc42f39c18d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:14.641994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53896f90-149a-4898-8f45-4767d41afbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:14.642677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53896f90-149a-4898-8f45-4767d41afbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:14.643006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53896f90-149a-4898-8f45-4767d41afbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:14.643282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53896f90-149a-4898-8f45-4767d41afbc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:14.657144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53896f90-149a-4898-8f45-4767d41afbc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:14.658770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53896f90-149a-4898-8f45-4767d41afbc9 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-53896f90-149a-4898-8f45-4767d41afbc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:14.681936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:14.683489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:14.684500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:14.684500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:14.684500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:14.684500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:14.684500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 623/1237] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:15:14.692096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209b9a4-a578-49f5-86b6-c72a2cfbcb81 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:14.841245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209b9a4-a578-49f5-86b6-c72a2cfbcb81 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:14.858038 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:14.896275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d7110-5c9d-4c86-b6c6-214226676705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:14.896868 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d7110-5c9d-4c86-b6c6-214226676705 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:14.897203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d7110-5c9d-4c86-b6c6-214226676705 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:14.897432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97d7110-5c9d-4c86-b6c6-214226676705 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:14.898181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d7110-5c9d-4c86-b6c6-214226676705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:14.898499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d7110-5c9d-4c86-b6c6-214226676705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:14.899967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97d7110-5c9d-4c86-b6c6-214226676705 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:14.915131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 615/1238] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:14.921241 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53896f90-149a-4898-8f45-4767d41afbc9 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:15.068644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53896f90-149a-4898-8f45-4767d41afbc9 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:15.124169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c318e29-1301-4ace-b385-f94ca89c467a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:15.126744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c318e29-1301-4ace-b385-f94ca89c467a None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:15.127183 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c318e29-1301-4ace-b385-f94ca89c467a None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:15.127469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c318e29-1301-4ace-b385-f94ca89c467a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:15.132847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:15.140661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c318e29-1301-4ace-b385-f94ca89c467a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:15.143607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c318e29-1301-4ace-b385-f94ca89c467a None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-1c318e29-1301-4ace-b385-f94ca89c467a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:15:15.162855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:15.164027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:15.165126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:15.165126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:15.165126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:15:15.165126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:15:15.165126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:15:15.165126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:15:15.165126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:15:15.165126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:15:15.165126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:15.165126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:15.165126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 624/1239] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:15.171442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d7110-5c9d-4c86-b6c6-214226676705 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:15.321338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97d7110-5c9d-4c86-b6c6-214226676705 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:15.375484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd18e35-9c29-4d27-a090-f46a0f6b229f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:15.376339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd18e35-9c29-4d27-a090-f46a0f6b229f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:15.376824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd18e35-9c29-4d27-a090-f46a0f6b229f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:15.377159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd18e35-9c29-4d27-a090-f46a0f6b229f None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:15.378560 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd18e35-9c29-4d27-a090-f46a0f6b229f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:15.378911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd18e35-9c29-4d27-a090-f46a0f6b229f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:15.381039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd18e35-9c29-4d27-a090-f46a0f6b229f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:15.405168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 616/1240] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Jul 20 08:15:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:15.414284 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c318e29-1301-4ace-b385-f94ca89c467a None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:15.577836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c318e29-1301-4ace-b385-f94ca89c467a None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:15.636761 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dc003-da1d-4ebf-bd4d-5cf9ee4ba043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:15.638371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dc003-da1d-4ebf-bd4d-5cf9ee4ba043 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:15.638758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dc003-da1d-4ebf-bd4d-5cf9ee4ba043 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:15.639126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576dc003-da1d-4ebf-bd4d-5cf9ee4ba043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:15.728214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:15.739095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576dc003-da1d-4ebf-bd4d-5cf9ee4ba043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:15.742096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576dc003-da1d-4ebf-bd4d-5cf9ee4ba043 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:15.889622 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 625/1241] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:15:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:15:15.898493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd18e35-9c29-4d27-a090-f46a0f6b229f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:15.966811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:16.069292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd18e35-9c29-4d27-a090-f46a0f6b229f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:16.139581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49011081-8059-49ec-80ef-33c4ebbee91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:16.140855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49011081-8059-49ec-80ef-33c4ebbee91f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:16.141274 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49011081-8059-49ec-80ef-33c4ebbee91f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:16.141635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49011081-8059-49ec-80ef-33c4ebbee91f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:16.156746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49011081-8059-49ec-80ef-33c4ebbee91f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:16.158283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49011081-8059-49ec-80ef-33c4ebbee91f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-49011081-8059-49ec-80ef-33c4ebbee91f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:16.181986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:16.183412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:16.184666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:16.184666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:16.184666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:16.184666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:16.184666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 617/1242] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:16.190312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dc003-da1d-4ebf-bd4d-5cf9ee4ba043 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:16.332027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:16.342135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576dc003-da1d-4ebf-bd4d-5cf9ee4ba043 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:16.401737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98c7af-3d1d-4e1d-a439-51f7dd8cbf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:16.402950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98c7af-3d1d-4e1d-a439-51f7dd8cbf94 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:16.403464 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98c7af-3d1d-4e1d-a439-51f7dd8cbf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:16.403960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98c7af-3d1d-4e1d-a439-51f7dd8cbf94 None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:16.405245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98c7af-3d1d-4e1d-a439-51f7dd8cbf94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:16.406542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98c7af-3d1d-4e1d-a439-51f7dd8cbf94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:16.408459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98c7af-3d1d-4e1d-a439-51f7dd8cbf94 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:16.426950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 626/1243] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:16.436149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49011081-8059-49ec-80ef-33c4ebbee91f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:16.576018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:16.611109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49011081-8059-49ec-80ef-33c4ebbee91f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:16.666881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb1862-8d94-488f-91c0-e91964918c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:16.668585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb1862-8d94-488f-91c0-e91964918c72 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:16.669277 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb1862-8d94-488f-91c0-e91964918c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:16.669636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb1862-8d94-488f-91c0-e91964918c72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:16.683152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb1862-8d94-488f-91c0-e91964918c72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:16.686016 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb1862-8d94-488f-91c0-e91964918c72 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-dcbb1862-8d94-488f-91c0-e91964918c72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:15:16.705275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:16.706384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:16.707370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:15:16.707370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:15:16.707370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:15:16.707370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:15:16.707370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:15:16.707370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:15:16.707370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:16.707370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:16.707370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 618/1244] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:15:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:16.713898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98c7af-3d1d-4e1d-a439-51f7dd8cbf94 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:16.840478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98c7af-3d1d-4e1d-a439-51f7dd8cbf94 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:16.882659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4f3c5-f7fe-478c-926a-c663fc3db038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:16.883314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4f3c5-f7fe-478c-926a-c663fc3db038 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:16.883595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4f3c5-f7fe-478c-926a-c663fc3db038 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:16.883865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4f3c5-f7fe-478c-926a-c663fc3db038 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:16.884696 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4f3c5-f7fe-478c-926a-c663fc3db038 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:16.884960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4f3c5-f7fe-478c-926a-c663fc3db038 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:16.886724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4f3c5-f7fe-478c-926a-c663fc3db038 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:16.909111 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 627/1245] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:15:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:16.910723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb1862-8d94-488f-91c0-e91964918c72 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:17.031619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb1862-8d94-488f-91c0-e91964918c72 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:17.087292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0417e8-f23a-4a34-937a-1783914f066f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:17.088205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0417e8-f23a-4a34-937a-1783914f066f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:17.088556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0417e8-f23a-4a34-937a-1783914f066f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:17.088932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0417e8-f23a-4a34-937a-1783914f066f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:17.089892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0417e8-f23a-4a34-937a-1783914f066f None admin] RBAC: Adding query filter params (role.id=d60298c676854e83b88df984e42773da, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=928054a577d14522bd0f00417f4f8c1f) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:17.090304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0417e8-f23a-4a34-937a-1783914f066f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:17.092030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0417e8-f23a-4a34-937a-1783914f066f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:17.101950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 619/1246] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:15:16 2022] GET /identity/v3/role_assignments?user.id=928054a577d14522bd0f00417f4f8c1f&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=d60298c676854e83b88df984e42773da => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:19.502191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032643d6-23b9-417b-8f90-98c84bd9b172 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:19.502829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032643d6-23b9-417b-8f90-98c84bd9b172 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:19.502829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032643d6-23b9-417b-8f90-98c84bd9b172 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:19.504412 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 628/1247] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:15:19.512040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ccd6-416d-462d-b8df-87dd3f1faea1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:19.512751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ccd6-416d-462d-b8df-87dd3f1faea1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:19.512751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc0ccd6-416d-462d-b8df-87dd3f1faea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:19.520362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:19.538514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8fc0ccd6-416d-462d-b8df-87dd3f1faea1 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:15:19.539841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8fc0ccd6-416d-462d-b8df-87dd3f1faea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:15:19.642905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 620/1248] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:15:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:15:19.670204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032643d6-23b9-417b-8f90-98c84bd9b172 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:19.692559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:19.792742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032643d6-23b9-417b-8f90-98c84bd9b172 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:19.842005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74e67b-09d1-4b68-ba9e-7bee02601efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:19.842005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74e67b-09d1-4b68-ba9e-7bee02601efd None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:19.842005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74e67b-09d1-4b68-ba9e-7bee02601efd None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:19.842005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74e67b-09d1-4b68-ba9e-7bee02601efd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:19.854708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74e67b-09d1-4b68-ba9e-7bee02601efd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-bc74e67b-09d1-4b68-ba9e-7bee02601efd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:15:19.862513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:19.863870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:19.865409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 629/1249] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:15:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:19.870503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0ccd6-416d-462d-b8df-87dd3f1faea1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:19.995670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc0ccd6-416d-462d-b8df-87dd3f1faea1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:20.037958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:20.044703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59a101-146e-4650-b9d9-fe327304c44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:20.045623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59a101-146e-4650-b9d9-fe327304c44b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:20.046021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59a101-146e-4650-b9d9-fe327304c44b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:20.046337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59a101-146e-4650-b9d9-fe327304c44b None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:20.047507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59a101-146e-4650-b9d9-fe327304c44b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:20.047855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59a101-146e-4650-b9d9-fe327304c44b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:20.049686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59a101-146e-4650-b9d9-fe327304c44b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:20.060807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 621/1250] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:15:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:20.067737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74e67b-09d1-4b68-ba9e-7bee02601efd None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:20.164290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:20.192088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74e67b-09d1-4b68-ba9e-7bee02601efd None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:20.237427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be85a2-dfad-4c43-9176-a3523b9dbfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:20.238591 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be85a2-dfad-4c43-9176-a3523b9dbfc3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:20.239089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be85a2-dfad-4c43-9176-a3523b9dbfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:20.239624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be85a2-dfad-4c43-9176-a3523b9dbfc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:20.253504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be85a2-dfad-4c43-9176-a3523b9dbfc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:20.255004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be85a2-dfad-4c43-9176-a3523b9dbfc3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-60be85a2-dfad-4c43-9176-a3523b9dbfc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:20.274893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:20.276034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:20.277039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:20.277039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:20.277039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:20.277039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:20.277039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 630/1251] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:20.283383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59a101-146e-4650-b9d9-fe327304c44b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:20.416826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59a101-146e-4650-b9d9-fe327304c44b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:20.470133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb6dd7-1072-4301-a6c4-5d31ef1244a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:20.471383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb6dd7-1072-4301-a6c4-5d31ef1244a3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:20.471871 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb6dd7-1072-4301-a6c4-5d31ef1244a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:20.472296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb6dd7-1072-4301-a6c4-5d31ef1244a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:20.473364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb6dd7-1072-4301-a6c4-5d31ef1244a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:20.473841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb6dd7-1072-4301-a6c4-5d31ef1244a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:20.475716 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb6dd7-1072-4301-a6c4-5d31ef1244a3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:20.489413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 622/1252] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:15:20.495292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be85a2-dfad-4c43-9176-a3523b9dbfc3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:20.634264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be85a2-dfad-4c43-9176-a3523b9dbfc3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:20.664645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:20.674376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a3b88-d078-4e44-a471-4b43d88782fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:20.675194 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a3b88-d078-4e44-a471-4b43d88782fa None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:20.675570 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a3b88-d078-4e44-a471-4b43d88782fa None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:20.675849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a3b88-d078-4e44-a471-4b43d88782fa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:20.689910 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a3b88-d078-4e44-a471-4b43d88782fa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:20.692352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a3b88-d078-4e44-a471-4b43d88782fa None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-7c4a3b88-d078-4e44-a471-4b43d88782fa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:15:20.709260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:20.710432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:20.711582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:20.711582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:20.711582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:15:20.711582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:15:20.711582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:15:20.711582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:15:20.711582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:15:20.711582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:15:20.711582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:20.711582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:20.711582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 631/1253] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:20.717077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb6dd7-1072-4301-a6c4-5d31ef1244a3 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:20.781494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:20.834334 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb6dd7-1072-4301-a6c4-5d31ef1244a3 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:20.876816 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf673df-613f-49c3-aa4b-dda635ea3691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:20.877507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf673df-613f-49c3-aa4b-dda635ea3691 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:20.877822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf673df-613f-49c3-aa4b-dda635ea3691 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:20.878077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf673df-613f-49c3-aa4b-dda635ea3691 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:20.878966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf673df-613f-49c3-aa4b-dda635ea3691 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:20.879275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf673df-613f-49c3-aa4b-dda635ea3691 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:20.880735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf673df-613f-49c3-aa4b-dda635ea3691 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:20.902311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 623/1254] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Jul 20 08:15:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:20.908944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a3b88-d078-4e44-a471-4b43d88782fa None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:21.051243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a3b88-d078-4e44-a471-4b43d88782fa None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:21.101767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93043b-3877-421e-99d6-60810e64c89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:21.102464 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93043b-3877-421e-99d6-60810e64c89b None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:21.102743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93043b-3877-421e-99d6-60810e64c89b None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:21.103026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93043b-3877-421e-99d6-60810e64c89b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:21.179627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:21.196693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93043b-3877-421e-99d6-60810e64c89b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:21.198222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93043b-3877-421e-99d6-60810e64c89b None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:21.324645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 632/1255] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:15:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:15:21.332654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf673df-613f-49c3-aa4b-dda635ea3691 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:21.474000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf673df-613f-49c3-aa4b-dda635ea3691 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:21.491831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:21.524848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d010630-aa40-4ecf-9c85-20051e67107b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:21.525752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d010630-aa40-4ecf-9c85-20051e67107b None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:21.526048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d010630-aa40-4ecf-9c85-20051e67107b None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:21.526306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d010630-aa40-4ecf-9c85-20051e67107b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:21.541196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d010630-aa40-4ecf-9c85-20051e67107b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:21.542790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d010630-aa40-4ecf-9c85-20051e67107b None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-6d010630-aa40-4ecf-9c85-20051e67107b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:21.563984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:21.565458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:21.566720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:21.566720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:21.566720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:21.566720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:21.566720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 624/1256] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:21.573668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93043b-3877-421e-99d6-60810e64c89b None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:21.678660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:21.694155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93043b-3877-421e-99d6-60810e64c89b None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:21.733456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e738d7-6d60-4ab9-98a6-750fb1b6e7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:21.735245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e738d7-6d60-4ab9-98a6-750fb1b6e7cf None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:21.735577 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e738d7-6d60-4ab9-98a6-750fb1b6e7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:21.735850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e738d7-6d60-4ab9-98a6-750fb1b6e7cf None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:21.736873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e738d7-6d60-4ab9-98a6-750fb1b6e7cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:21.737268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e738d7-6d60-4ab9-98a6-750fb1b6e7cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:21.738818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e738d7-6d60-4ab9-98a6-750fb1b6e7cf None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:21.754541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 633/1257] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:21.763247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d010630-aa40-4ecf-9c85-20051e67107b None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:21.885564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d010630-aa40-4ecf-9c85-20051e67107b None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:21.932440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90234453-6456-4b97-ae5f-1f457aa849dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:21.933156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90234453-6456-4b97-ae5f-1f457aa849dc None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:21.933439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90234453-6456-4b97-ae5f-1f457aa849dc None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:21.933716 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90234453-6456-4b97-ae5f-1f457aa849dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:21.941009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90234453-6456-4b97-ae5f-1f457aa849dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:21.942505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90234453-6456-4b97-ae5f-1f457aa849dc None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-90234453-6456-4b97-ae5f-1f457aa849dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:15:21.956843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:21.958194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:21.959812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:15:21.959812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:15:21.959812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:15:21.959812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:15:21.959812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:15:21.959812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:15:21.959812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:21.959812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:21.959812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 625/1258] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:15:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:15:21.964353 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e738d7-6d60-4ab9-98a6-750fb1b6e7cf None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:22.086824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e738d7-6d60-4ab9-98a6-750fb1b6e7cf None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:22.123710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063a675-95c9-46a3-8d80-c21c5dc05c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:22.124985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063a675-95c9-46a3-8d80-c21c5dc05c28 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:22.125399 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063a675-95c9-46a3-8d80-c21c5dc05c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:22.125675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063a675-95c9-46a3-8d80-c21c5dc05c28 None admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:22.126515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063a675-95c9-46a3-8d80-c21c5dc05c28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:22.126886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063a675-95c9-46a3-8d80-c21c5dc05c28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:22.128666 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063a675-95c9-46a3-8d80-c21c5dc05c28 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:22.144674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 634/1259] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:15:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:22.150928 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90234453-6456-4b97-ae5f-1f457aa849dc None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:22.181408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:22.274713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90234453-6456-4b97-ae5f-1f457aa849dc None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:22.315153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f795a8-a6cb-4e92-a763-fe14707e63eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:22.315812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f795a8-a6cb-4e92-a763-fe14707e63eb None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:22.316097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f795a8-a6cb-4e92-a763-fe14707e63eb None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:22.316372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f795a8-a6cb-4e92-a763-fe14707e63eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:22.317203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f795a8-a6cb-4e92-a763-fe14707e63eb None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=928054a577d14522bd0f00417f4f8c1f) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:22.317521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f795a8-a6cb-4e92-a763-fe14707e63eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:22.318997 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f795a8-a6cb-4e92-a763-fe14707e63eb None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:22.324938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 626/1260] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:15:22 2022] GET /identity/v3/role_assignments?user.id=928054a577d14522bd0f00417f4f8c1f&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:24.464543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05cb7-ef04-454f-b217-eb49359c7599 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:24.465138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05cb7-ef04-454f-b217-eb49359c7599 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:24.465138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c05cb7-ef04-454f-b217-eb49359c7599 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:24.466475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 635/1261] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:15:24.471827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd3be-6b6b-45ed-923b-2f696ee9d639 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:24.472350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd3be-6b6b-45ed-923b-2f696ee9d639 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:24.472453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7bd3be-6b6b-45ed-923b-2f696ee9d639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:24.489615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1b7bd3be-6b6b-45ed-923b-2f696ee9d639 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:15:24.490864 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1b7bd3be-6b6b-45ed-923b-2f696ee9d639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:15:24.580563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 627/1262] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:15:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:15:24.608480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c05cb7-ef04-454f-b217-eb49359c7599 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:24.620945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:24.715752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c05cb7-ef04-454f-b217-eb49359c7599 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:24.751740 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72448dd5-7840-43a2-b9d2-cf0cee25a086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:24.752522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72448dd5-7840-43a2-b9d2-cf0cee25a086 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:24.752798 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72448dd5-7840-43a2-b9d2-cf0cee25a086 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:24.753067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72448dd5-7840-43a2-b9d2-cf0cee25a086 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:24.763718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72448dd5-7840-43a2-b9d2-cf0cee25a086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-72448dd5-7840-43a2-b9d2-cf0cee25a086 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:15:24.771454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:24.772703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:24.773945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 636/1263] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:15:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:24.779268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bd3be-6b6b-45ed-923b-2f696ee9d639 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:24.798266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:24.886016 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7bd3be-6b6b-45ed-923b-2f696ee9d639 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:24.924677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af51890-de92-48da-9e55-08733cca7e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:24.925354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af51890-de92-48da-9e55-08733cca7e77 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:24.925645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af51890-de92-48da-9e55-08733cca7e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:24.925926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af51890-de92-48da-9e55-08733cca7e77 None admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:24.926712 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af51890-de92-48da-9e55-08733cca7e77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:24.927028 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af51890-de92-48da-9e55-08733cca7e77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:24.928411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af51890-de92-48da-9e55-08733cca7e77 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:24.937847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 628/1264] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:15:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:24.943929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72448dd5-7840-43a2-b9d2-cf0cee25a086 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:25.011885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:25.046715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72448dd5-7840-43a2-b9d2-cf0cee25a086 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:25.081062 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4c219-5450-4fb0-ac60-2d18921d5e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:25.081859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4c219-5450-4fb0-ac60-2d18921d5e03 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:25.082145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4c219-5450-4fb0-ac60-2d18921d5e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:25.082421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4c219-5450-4fb0-ac60-2d18921d5e03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:25.093374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4c219-5450-4fb0-ac60-2d18921d5e03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:25.094868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4c219-5450-4fb0-ac60-2d18921d5e03 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-52c4c219-5450-4fb0-ac60-2d18921d5e03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:25.111686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:25.112745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:25.113916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:25.113916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:25.113916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:25.113916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:25.113916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 637/1265] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:25.119062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af51890-de92-48da-9e55-08733cca7e77 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:25.203929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:25.224639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af51890-de92-48da-9e55-08733cca7e77 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:25.264095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ae4c9-3f8e-4610-af8e-10e59a6d8498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:25.264696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae4c9-3f8e-4610-af8e-10e59a6d8498 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:25.264982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae4c9-3f8e-4610-af8e-10e59a6d8498 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:25.265791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ae4c9-3f8e-4610-af8e-10e59a6d8498 None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:25.266530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ae4c9-3f8e-4610-af8e-10e59a6d8498 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:25.266875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ae4c9-3f8e-4610-af8e-10e59a6d8498 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:25.268284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ae4c9-3f8e-4610-af8e-10e59a6d8498 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:25.282489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 629/1266] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:25.288448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4c219-5450-4fb0-ac60-2d18921d5e03 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:25.410416 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4c219-5450-4fb0-ac60-2d18921d5e03 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:25.441054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:25.459762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178556a-91b9-4617-b115-7b0346664955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:25.460601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178556a-91b9-4617-b115-7b0346664955 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:25.460940 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178556a-91b9-4617-b115-7b0346664955 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:25.461322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178556a-91b9-4617-b115-7b0346664955 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:25.470509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178556a-91b9-4617-b115-7b0346664955 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:25.472027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178556a-91b9-4617-b115-7b0346664955 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c178556a-91b9-4617-b115-7b0346664955 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:15:25.487694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:25.488922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:25.489914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:25.489914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:25.489914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:15:25.489914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:15:25.489914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:15:25.489914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:15:25.489914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:15:25.489914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:15:25.489914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:25.489914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:25.489914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 638/1267] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:25.495420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ae4c9-3f8e-4610-af8e-10e59a6d8498 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:25.624960 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ae4c9-3f8e-4610-af8e-10e59a6d8498 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:25.662494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeaac63-219c-4b1e-89dd-58904c576429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:25.663206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaac63-219c-4b1e-89dd-58904c576429 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:25.663492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaac63-219c-4b1e-89dd-58904c576429 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:25.663767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeaac63-219c-4b1e-89dd-58904c576429 None admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:25.664620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeaac63-219c-4b1e-89dd-58904c576429 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:25.664957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeaac63-219c-4b1e-89dd-58904c576429 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:25.666397 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeaac63-219c-4b1e-89dd-58904c576429 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:25.681476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:25.686785 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 630/1268] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Jul 20 08:15:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:25.693284 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178556a-91b9-4617-b115-7b0346664955 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:25.806362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178556a-91b9-4617-b115-7b0346664955 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:25.844521 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ceccc-2c8a-4599-9648-42c673d95f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:25.845321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ceccc-2c8a-4599-9648-42c673d95f6d None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:25.845608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ceccc-2c8a-4599-9648-42c673d95f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:25.845887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ceccc-2c8a-4599-9648-42c673d95f6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:25.863631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ceccc-2c8a-4599-9648-42c673d95f6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:25.865238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ceccc-2c8a-4599-9648-42c673d95f6d None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-0c8ceccc-2c8a-4599-9648-42c673d95f6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:25.883763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:25.884810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:25.886323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:25.886323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:25.886323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:25.886323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:25.886323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 639/1269] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:25.892390 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeaac63-219c-4b1e-89dd-58904c576429 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:26.004183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeaac63-219c-4b1e-89dd-58904c576429 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:26.042743 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1bade-cbc0-4df5-9a8e-d1ca5abb9d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:26.043483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1bade-cbc0-4df5-9a8e-d1ca5abb9d3f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:26.043771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1bade-cbc0-4df5-9a8e-d1ca5abb9d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:26.044051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1bade-cbc0-4df5-9a8e-d1ca5abb9d3f None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:26.044789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1bade-cbc0-4df5-9a8e-d1ca5abb9d3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:26.045098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1bade-cbc0-4df5-9a8e-d1ca5abb9d3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:26.046558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1bade-cbc0-4df5-9a8e-d1ca5abb9d3f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:26.059089 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 631/1270] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:26.067029 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ceccc-2c8a-4599-9648-42c673d95f6d None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:26.101076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:26.189940 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ceccc-2c8a-4599-9648-42c673d95f6d None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:26.231762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56604a7-fb62-4241-a516-61e69e2bd8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:26.232564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56604a7-fb62-4241-a516-61e69e2bd8fa None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:26.232851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56604a7-fb62-4241-a516-61e69e2bd8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:26.233130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56604a7-fb62-4241-a516-61e69e2bd8fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:26.240887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56604a7-fb62-4241-a516-61e69e2bd8fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:26.242669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56604a7-fb62-4241-a516-61e69e2bd8fa None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-c56604a7-fb62-4241-a516-61e69e2bd8fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:15:26.257959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:26.259163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:26.260376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:15:26.260376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:15:26.260376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:15:26.260376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:15:26.260376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:15:26.260376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:15:26.260376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:26.260376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:26.260376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 640/1271] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:15:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:26.266092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1bade-cbc0-4df5-9a8e-d1ca5abb9d3f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:26.336885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:26.383272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1bade-cbc0-4df5-9a8e-d1ca5abb9d3f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:26.431688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8ea22-6596-448a-b2a6-e8c8701aec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:26.432460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8ea22-6596-448a-b2a6-e8c8701aec7c None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:26.432756 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8ea22-6596-448a-b2a6-e8c8701aec7c None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:26.433046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8ea22-6596-448a-b2a6-e8c8701aec7c None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:26.433868 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8ea22-6596-448a-b2a6-e8c8701aec7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:26.434191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8ea22-6596-448a-b2a6-e8c8701aec7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:26.435697 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8ea22-6596-448a-b2a6-e8c8701aec7c None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:26.449303 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 632/1272] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:15:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:15:26.457008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56604a7-fb62-4241-a516-61e69e2bd8fa None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:26.565824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:26.575911 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56604a7-fb62-4241-a516-61e69e2bd8fa None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:26.614640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d2d57d-6307-4537-8595-dfdbff305074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:26.615549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2d57d-6307-4537-8595-dfdbff305074 None admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:26.615837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2d57d-6307-4537-8595-dfdbff305074 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:26.616118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d2d57d-6307-4537-8595-dfdbff305074 None admin] PATH_INFO: `/v3/projects/9a4d252038c649faadd6e91924ffb297/users/928054a577d14522bd0f00417f4f8c1f/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:26.618429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2d57d-6307-4537-8595-dfdbff305074 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a4d252038c649faadd6e91924ffb297, user_id=928054a577d14522bd0f00417f4f8c1f, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:26.619935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d2d57d-6307-4537-8595-dfdbff305074 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:26.634571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 641/1273] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Jul 20 08:15:26 2022] PUT /identity/v3/projects/9a4d252038c649faadd6e91924ffb297/users/928054a577d14522bd0f00417f4f8c1f/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:15:28.719428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291c20c-06d1-42f7-b8ea-7c8740a4b1c8 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:28.719994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291c20c-06d1-42f7-b8ea-7c8740a4b1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:28.719994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291c20c-06d1-42f7-b8ea-7c8740a4b1c8 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:28.721334 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 633/1274] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:15:28.726764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209c8d9-49a5-4259-a1ff-3191ad8a86bb None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:28.727194 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209c8d9-49a5-4259-a1ff-3191ad8a86bb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:28.727296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9209c8d9-49a5-4259-a1ff-3191ad8a86bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:28.744732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9209c8d9-49a5-4259-a1ff-3191ad8a86bb None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:15:28.745937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9209c8d9-49a5-4259-a1ff-3191ad8a86bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:15:28.816345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 642/1275] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:15:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:15:28.841687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291c20c-06d1-42f7-b8ea-7c8740a4b1c8 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:28.957831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291c20c-06d1-42f7-b8ea-7c8740a4b1c8 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:28.974918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:28.992308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca220875-2f69-41c6-a9ba-ee3a7f3516c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:28.993096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca220875-2f69-41c6-a9ba-ee3a7f3516c4 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:28.993391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca220875-2f69-41c6-a9ba-ee3a7f3516c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:28.993660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca220875-2f69-41c6-a9ba-ee3a7f3516c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:29.004424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca220875-2f69-41c6-a9ba-ee3a7f3516c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-ca220875-2f69-41c6-a9ba-ee3a7f3516c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 08:15:29.012017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:29.013072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:29.013936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 634/1276] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Jul 20 08:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:29.019639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209c8d9-49a5-4259-a1ff-3191ad8a86bb None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:29.103284 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:29.122997 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9209c8d9-49a5-4259-a1ff-3191ad8a86bb None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:29.157950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456556e1-01db-41b2-8a55-6e80ad8821a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:29.158661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456556e1-01db-41b2-8a55-6e80ad8821a1 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:29.158951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456556e1-01db-41b2-8a55-6e80ad8821a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:29.159228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456556e1-01db-41b2-8a55-6e80ad8821a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:29.159999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456556e1-01db-41b2-8a55-6e80ad8821a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:29.160314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456556e1-01db-41b2-8a55-6e80ad8821a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:29.161732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456556e1-01db-41b2-8a55-6e80ad8821a1 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:29.171232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 643/1277] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Jul 20 08:15:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:29.177968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca220875-2f69-41c6-a9ba-ee3a7f3516c4 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:29.290675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca220875-2f69-41c6-a9ba-ee3a7f3516c4 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:29.331699 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f9924-1b0c-4cbe-9553-40f2029fb3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:29.332498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f9924-1b0c-4cbe-9553-40f2029fb3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:29.332790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f9924-1b0c-4cbe-9553-40f2029fb3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:29.333064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f9924-1b0c-4cbe-9553-40f2029fb3c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:29.345066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f9924-1b0c-4cbe-9553-40f2029fb3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:29.346523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079f9924-1b0c-4cbe-9553-40f2029fb3c9 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-079f9924-1b0c-4cbe-9553-40f2029fb3c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:29.364938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:29.366094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:29.367564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:29.367564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:29.367564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:29.367564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:29.367564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 635/1278] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:29.372844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456556e1-01db-41b2-8a55-6e80ad8821a1 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:29.495100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456556e1-01db-41b2-8a55-6e80ad8821a1 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:29.532985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f60c20-ab6f-422c-a1d6-3ce929d9baab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:29.533659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f60c20-ab6f-422c-a1d6-3ce929d9baab None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:29.533942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f60c20-ab6f-422c-a1d6-3ce929d9baab None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:29.534235 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f60c20-ab6f-422c-a1d6-3ce929d9baab None admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:29.535003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f60c20-ab6f-422c-a1d6-3ce929d9baab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:29.535319 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f60c20-ab6f-422c-a1d6-3ce929d9baab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:29.536806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f60c20-ab6f-422c-a1d6-3ce929d9baab None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:29.552049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 644/1279] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:29.558810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f9924-1b0c-4cbe-9553-40f2029fb3c9 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:29.614790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:29.727329 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f9924-1b0c-4cbe-9553-40f2029fb3c9 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:29.765963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017b828-8049-41e1-a444-307f5c8d648f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:29.766804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017b828-8049-41e1-a444-307f5c8d648f None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:29.767094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017b828-8049-41e1-a444-307f5c8d648f None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:29.767368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017b828-8049-41e1-a444-307f5c8d648f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:29.777761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017b828-8049-41e1-a444-307f5c8d648f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:29.779293 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017b828-8049-41e1-a444-307f5c8d648f None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-d017b828-8049-41e1-a444-307f5c8d648f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 08:15:29.794968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:29.796773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:29.798905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:29.798905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:29.798905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 08:15:29.798905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 08:15:29.798905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 08:15:29.798905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 08:15:29.798905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 08:15:29.798905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 08:15:29.798905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 08:15:29.798905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:29.798905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 636/1280] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:29.803945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f60c20-ab6f-422c-a1d6-3ce929d9baab None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:29.808391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:29.914743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f60c20-ab6f-422c-a1d6-3ce929d9baab None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:29.955026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535f78a-310f-48d8-b444-4c81550a127e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:29.955707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535f78a-310f-48d8-b444-4c81550a127e None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:29.955988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535f78a-310f-48d8-b444-4c81550a127e None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:29.956266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535f78a-310f-48d8-b444-4c81550a127e None admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:29.957118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535f78a-310f-48d8-b444-4c81550a127e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:29.957424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535f78a-310f-48d8-b444-4c81550a127e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:29.958917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535f78a-310f-48d8-b444-4c81550a127e None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:29.977474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 645/1281] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Jul 20 08:15:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:15:29.983415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017b828-8049-41e1-a444-307f5c8d648f None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:30.086802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:30.097932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017b828-8049-41e1-a444-307f5c8d648f None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:30.136498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3ca50-2b65-495d-81e6-2de6a6b961ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:30.137179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3ca50-2b65-495d-81e6-2de6a6b961ce None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:30.137475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3ca50-2b65-495d-81e6-2de6a6b961ce None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:30.137756 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3ca50-2b65-495d-81e6-2de6a6b961ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:30.208528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3ca50-2b65-495d-81e6-2de6a6b961ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:30.209996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3ca50-2b65-495d-81e6-2de6a6b961ce None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:30.307134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:30.319331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 637/1282] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Jul 20 08:15:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 08:15:30.326258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535f78a-310f-48d8-b444-4c81550a127e None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:30.403386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:30.445096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535f78a-310f-48d8-b444-4c81550a127e None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:30.480856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ae8a6-a0fa-49b4-b1d3-c6c1e9e53aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:30.481654 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ae8a6-a0fa-49b4-b1d3-c6c1e9e53aca None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:30.481937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ae8a6-a0fa-49b4-b1d3-c6c1e9e53aca None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:30.482221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ae8a6-a0fa-49b4-b1d3-c6c1e9e53aca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:30.495468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ae8a6-a0fa-49b4-b1d3-c6c1e9e53aca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:30.496951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098ae8a6-a0fa-49b4-b1d3-c6c1e9e53aca None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-098ae8a6-a0fa-49b4-b1d3-c6c1e9e53aca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 08:15:30.515724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:30.516941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:30.518503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 08:15:30.518503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 08:15:30.518503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 08:15:30.518503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:30.518503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 646/1283] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:30.523143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3ca50-2b65-495d-81e6-2de6a6b961ce None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:30.625391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3ca50-2b65-495d-81e6-2de6a6b961ce None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:30.661462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2182a9-f6db-4e16-9c82-3239caf47aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:30.662038 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2182a9-f6db-4e16-9c82-3239caf47aef None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:30.662323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2182a9-f6db-4e16-9c82-3239caf47aef None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:30.662622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2182a9-f6db-4e16-9c82-3239caf47aef None admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:30.663348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2182a9-f6db-4e16-9c82-3239caf47aef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:30.663661 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2182a9-f6db-4e16-9c82-3239caf47aef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:30.665049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2182a9-f6db-4e16-9c82-3239caf47aef None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:30.682568 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 638/1284] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Jul 20 08:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:30.688840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ae8a6-a0fa-49b4-b1d3-c6c1e9e53aca None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:30.805773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ae8a6-a0fa-49b4-b1d3-c6c1e9e53aca None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:30.823657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:30.848281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6d3f8-c797-44a3-8395-c00697c5bda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:30.849102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6d3f8-c797-44a3-8395-c00697c5bda5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:30.849387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6d3f8-c797-44a3-8395-c00697c5bda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:30.849667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6d3f8-c797-44a3-8395-c00697c5bda5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:30.857455 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6d3f8-c797-44a3-8395-c00697c5bda5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:30.859085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6d3f8-c797-44a3-8395-c00697c5bda5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application [None req-3fa6d3f8-c797-44a3-8395-c00697c5bda5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 08:15:30.873632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:30.875543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:30.876913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 08:15:30.876913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 08:15:30.876913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 08:15:30.876913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 08:15:30.876913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 08:15:30.876913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 08:15:30.876913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 08:15:30.876913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: ERROR keystone.server.flask.application Jul 20 08:15:30.876913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 647/1285] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Jul 20 08:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:30.881568 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2182a9-f6db-4e16-9c82-3239caf47aef None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:30.935733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:30.991311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2182a9-f6db-4e16-9c82-3239caf47aef None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:31.027397 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187cdf5-fc4e-48ba-b3e3-8d4252bacaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:31.028077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187cdf5-fc4e-48ba-b3e3-8d4252bacaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:31.028368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187cdf5-fc4e-48ba-b3e3-8d4252bacaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:31.028647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187cdf5-fc4e-48ba-b3e3-8d4252bacaf3 None admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:31.029454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187cdf5-fc4e-48ba-b3e3-8d4252bacaf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:31.029768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187cdf5-fc4e-48ba-b3e3-8d4252bacaf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:31.032270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9187cdf5-fc4e-48ba-b3e3-8d4252bacaf3 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:31.045471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 639/1286] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Jul 20 08:15:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:31.052578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6d3f8-c797-44a3-8395-c00697c5bda5 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:31.160858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6d3f8-c797-44a3-8395-c00697c5bda5 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:31.196200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c9aeb-4f0c-4284-aaed-1e790e644794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:31.196761 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9aeb-4f0c-4284-aaed-1e790e644794 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:31.197045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9aeb-4f0c-4284-aaed-1e790e644794 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:31.197317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c9aeb-4f0c-4284-aaed-1e790e644794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:31.198148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c9aeb-4f0c-4284-aaed-1e790e644794 None admin] RBAC: Adding query filter params (role.id=a48bc1eaca7e4ade827ca5b68110af29, scope.project.id=9a4d252038c649faadd6e91924ffb297, user.id=928054a577d14522bd0f00417f4f8c1f) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:31.198454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c9aeb-4f0c-4284-aaed-1e790e644794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:31.199947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c9aeb-4f0c-4284-aaed-1e790e644794 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:31.206943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 648/1287] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Jul 20 08:15:31 2022] GET /identity/v3/role_assignments?user.id=928054a577d14522bd0f00417f4f8c1f&scope.project.id=9a4d252038c649faadd6e91924ffb297&role.id=a48bc1eaca7e4ade827ca5b68110af29 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:33.333832 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e528b-b784-4ff2-96d8-674edee85ca6 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:33.334402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e528b-b784-4ff2-96d8-674edee85ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:33.334402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e528b-b784-4ff2-96d8-674edee85ca6 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:33.335796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 640/1288] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:15:33.341406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bf69e-339e-4853-9f7e-34d2446ae26d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:33.342138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bf69e-339e-4853-9f7e-34d2446ae26d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:33.342138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bf69e-339e-4853-9f7e-34d2446ae26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:33.359275 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a65bf69e-339e-4853-9f7e-34d2446ae26d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:15:33.360726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a65bf69e-339e-4853-9f7e-34d2446ae26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:15:33.400086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:33.452334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 649/1289] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:15:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:15:33.485917 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e528b-b784-4ff2-96d8-674edee85ca6 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:33.587833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e528b-b784-4ff2-96d8-674edee85ca6 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:33.598438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:33.622831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33be4f7-07d5-4df7-9df4-09918fe2219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:33.623632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33be4f7-07d5-4df7-9df4-09918fe2219a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:33.623918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33be4f7-07d5-4df7-9df4-09918fe2219a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:33.624195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33be4f7-07d5-4df7-9df4-09918fe2219a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:33.625419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33be4f7-07d5-4df7-9df4-09918fe2219a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:33.626946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33be4f7-07d5-4df7-9df4-09918fe2219a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-b33be4f7-07d5-4df7-9df4-09918fe2219a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:33.639101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:33.640350 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:33.641257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 641/1290] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Jul 20 08:15:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:33.646922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bf69e-339e-4853-9f7e-34d2446ae26d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:33.751248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bf69e-339e-4853-9f7e-34d2446ae26d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:33.789667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace9690-df1e-42cb-a665-be57c6df4eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:33.790326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace9690-df1e-42cb-a665-be57c6df4eaa None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:33.790633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace9690-df1e-42cb-a665-be57c6df4eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:33.790911 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace9690-df1e-42cb-a665-be57c6df4eaa None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:33.791618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace9690-df1e-42cb-a665-be57c6df4eaa None admin] RBAC: Adding query filter params (name=placement) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:33.791922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace9690-df1e-42cb-a665-be57c6df4eaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:33.793317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace9690-df1e-42cb-a665-be57c6df4eaa None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:33.799436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 650/1291] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Jul 20 08:15:33 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:33.805395 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33be4f7-07d5-4df7-9df4-09918fe2219a None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:33.919277 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33be4f7-07d5-4df7-9df4-09918fe2219a None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:33.959038 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba947ae-5a9b-4e5e-9c9e-fb2016fa9f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:33.959713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba947ae-5a9b-4e5e-9c9e-fb2016fa9f59 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:33.960001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba947ae-5a9b-4e5e-9c9e-fb2016fa9f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:33.960283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba947ae-5a9b-4e5e-9c9e-fb2016fa9f59 None admin] PATH_INFO: `/v3/services` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:33.961009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba947ae-5a9b-4e5e-9c9e-fb2016fa9f59 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:33.961315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba947ae-5a9b-4e5e-9c9e-fb2016fa9f59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:33.962735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba947ae-5a9b-4e5e-9c9e-fb2016fa9f59 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:33.967830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 642/1292] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Jul 20 08:15:33 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:36.045010 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da70fb-a521-478a-ad76-503218c7b786 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:36.045535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da70fb-a521-478a-ad76-503218c7b786 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:36.045633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da70fb-a521-478a-ad76-503218c7b786 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:36.047571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 651/1293] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:15:36.053079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be629d8-5b55-4428-ad30-7a3342f2c125 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:36.053552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be629d8-5b55-4428-ad30-7a3342f2c125 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:36.053647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be629d8-5b55-4428-ad30-7a3342f2c125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:36.070423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3be629d8-5b55-4428-ad30-7a3342f2c125 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:15:36.071678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3be629d8-5b55-4428-ad30-7a3342f2c125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:15:36.102445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:36.143619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 643/1294] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:15:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:15:36.166803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da70fb-a521-478a-ad76-503218c7b786 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:36.201847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:36.276621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da70fb-a521-478a-ad76-503218c7b786 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:36.311304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c04b480-62c0-46ab-b471-0ff9335b5445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:36.311991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04b480-62c0-46ab-b471-0ff9335b5445 None admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:36.312361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04b480-62c0-46ab-b471-0ff9335b5445 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:36.312642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c04b480-62c0-46ab-b471-0ff9335b5445 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:36.313375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04b480-62c0-46ab-b471-0ff9335b5445 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:36.314774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c04b480-62c0-46ab-b471-0ff9335b5445 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:36.328177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 652/1295] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Jul 20 08:15:36 2022] POST /identity/v3/services => generated 246 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:15:38.396297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6b35c-04cf-4e78-b112-7665a281c4e0 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:38.396867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6b35c-04cf-4e78-b112-7665a281c4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:38.396867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d6b35c-04cf-4e78-b112-7665a281c4e0 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:38.398287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 644/1296] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:15:38.404105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95a918-4747-4bdc-865a-6dc304d2b04c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:38.404584 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95a918-4747-4bdc-865a-6dc304d2b04c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:38.404677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95a918-4747-4bdc-865a-6dc304d2b04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:38.425149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8c95a918-4747-4bdc-865a-6dc304d2b04c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:15:38.426453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8c95a918-4747-4bdc-865a-6dc304d2b04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:15:38.505618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 653/1297] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:15:38 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:15:38.528794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6b35c-04cf-4e78-b112-7665a281c4e0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:38.639464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d6b35c-04cf-4e78-b112-7665a281c4e0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:38.678514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01510a-c980-43d0-9e5c-6f64c0d024e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:38.679279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01510a-c980-43d0-9e5c-6f64c0d024e7 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:38.679565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01510a-c980-43d0-9e5c-6f64c0d024e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:38.679843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f01510a-c980-43d0-9e5c-6f64c0d024e7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:38.680508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01510a-c980-43d0-9e5c-6f64c0d024e7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:38.681906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f01510a-c980-43d0-9e5c-6f64c0d024e7 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-0f01510a-c980-43d0-9e5c-6f64c0d024e7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:38.695529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:38.696836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:38.698695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 645/1298] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Jul 20 08:15:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 08:15:38.705601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95a918-4747-4bdc-865a-6dc304d2b04c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:38.724780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:38.815044 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95a918-4747-4bdc-865a-6dc304d2b04c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:38.851019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903adcc-208a-42a3-b083-510eb032c4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:38.851690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903adcc-208a-42a3-b083-510eb032c4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:38.851978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903adcc-208a-42a3-b083-510eb032c4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:38.852257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5903adcc-208a-42a3-b083-510eb032c4c3 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:38.852976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903adcc-208a-42a3-b083-510eb032c4c3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:38.853289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903adcc-208a-42a3-b083-510eb032c4c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:38.854699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5903adcc-208a-42a3-b083-510eb032c4c3 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:38.860297 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 654/1299] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Jul 20 08:15:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:15:38.866373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01510a-c980-43d0-9e5c-6f64c0d024e7 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:38.899656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:38.973338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f01510a-c980-43d0-9e5c-6f64c0d024e7 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:39.008079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cf2c-b371-4ea4-9a62-166258d9837a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:39.008671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cf2c-b371-4ea4-9a62-166258d9837a None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:39.008961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cf2c-b371-4ea4-9a62-166258d9837a None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:39.009241 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cf2c-b371-4ea4-9a62-166258d9837a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:39.010050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cf2c-b371-4ea4-9a62-166258d9837a None admin] RBAC: Adding query filter params (interface=public, service_id=ab82edf70ea340a2bf82cc780630ef55, region_id=RegionOne) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:39.010364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cf2c-b371-4ea4-9a62-166258d9837a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:39.011793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cf2c-b371-4ea4-9a62-166258d9837a None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:39.018060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 646/1300] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Jul 20 08:15:38 2022] GET /identity/v3/endpoints?service_id=ab82edf70ea340a2bf82cc780630ef55&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:39.023883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903adcc-208a-42a3-b083-510eb032c4c3 None admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:39.111906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:39.130867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5903adcc-208a-42a3-b083-510eb032c4c3 None admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:39.168177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4232bc-208a-41c2-b2fd-2814a3a35432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:39.168852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4232bc-208a-41c2-b2fd-2814a3a35432 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:39.169136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4232bc-208a-41c2-b2fd-2814a3a35432 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:39.169482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4232bc-208a-41c2-b2fd-2814a3a35432 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:39.170185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4232bc-208a-41c2-b2fd-2814a3a35432 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:39.171543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4232bc-208a-41c2-b2fd-2814a3a35432 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:39.179322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 655/1301] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Jul 20 08:15:39 2022] GET /identity/v3/services => generated 2191 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:15:41.230611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876690d1-3634-4bd2-8750-b78d66fe6690 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:41.231188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876690d1-3634-4bd2-8750-b78d66fe6690 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:41.231188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876690d1-3634-4bd2-8750-b78d66fe6690 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:41.232552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 647/1302] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:15:41.238197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199793f-cee4-4c01-b52a-3e2b94bd2055 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:41.238667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199793f-cee4-4c01-b52a-3e2b94bd2055 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:41.238667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199793f-cee4-4c01-b52a-3e2b94bd2055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:41.256755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c199793f-cee4-4c01-b52a-3e2b94bd2055 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:15:41.257961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c199793f-cee4-4c01-b52a-3e2b94bd2055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:15:41.343514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 656/1303] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:15:41 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 08:15:41.367286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876690d1-3634-4bd2-8750-b78d66fe6690 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:41.465402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:41.472739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876690d1-3634-4bd2-8750-b78d66fe6690 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:41.508539 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79c51a-384b-4bdf-bbff-fab85895ff42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:41.509351 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79c51a-384b-4bdf-bbff-fab85895ff42 None admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:41.509638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79c51a-384b-4bdf-bbff-fab85895ff42 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:41.509937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d79c51a-384b-4bdf-bbff-fab85895ff42 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:41.510573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79c51a-384b-4bdf-bbff-fab85895ff42 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:41.511984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d79c51a-384b-4bdf-bbff-fab85895ff42 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application [None req-1d79c51a-384b-4bdf-bbff-fab85895ff42 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 08:15:41.524781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 08:15:41.526001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application with Lock( Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self._enter() Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application return self.creator() Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application created_value = creator( Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: ERROR keystone.server.flask.application Jul 20 08:15:41.527164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 648/1304] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Jul 20 08:15:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 08:15:41.533158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199793f-cee4-4c01-b52a-3e2b94bd2055 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:41.619917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:41.645043 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199793f-cee4-4c01-b52a-3e2b94bd2055 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:41.688837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed85ea6-edcd-4c33-8433-6a2ad353edf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:41.689638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85ea6-edcd-4c33-8433-6a2ad353edf5 None admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:41.689965 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85ea6-edcd-4c33-8433-6a2ad353edf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:41.690290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed85ea6-edcd-4c33-8433-6a2ad353edf5 None admin] PATH_INFO: `/v3/services` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:41.691093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed85ea6-edcd-4c33-8433-6a2ad353edf5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:15:41.691444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed85ea6-edcd-4c33-8433-6a2ad353edf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:41.693013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed85ea6-edcd-4c33-8433-6a2ad353edf5 None admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:41.702021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 657/1305] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Jul 20 08:15:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:15:41.709056 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79c51a-384b-4bdf-bbff-fab85895ff42 None admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:15:41.810989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d79c51a-384b-4bdf-bbff-fab85895ff42 None admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:15:41.846523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a2827-adab-48c6-941d-2964d3ef5d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:15:41.847205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a2827-adab-48c6-941d-2964d3ef5d52 None admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:15:41.847489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a2827-adab-48c6-941d-2964d3ef5d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:15:41.847765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a2827-adab-48c6-941d-2964d3ef5d52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:15:41.848512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a2827-adab-48c6-941d-2964d3ef5d52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:15:41.849906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5a2827-adab-48c6-941d-2964d3ef5d52 None admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:15:41.868313 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 649/1306] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Jul 20 08:15:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:15:51.696255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=30 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:15:51.863791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:16:53.131771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c8db6-2dda-4807-bf65-0698510a4da2 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:16:53.132602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c8db6-2dda-4807-bf65-0698510a4da2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:16:53.132602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c8db6-2dda-4807-bf65-0698510a4da2 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:16:53.133763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 658/1307] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:16:53.145703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590df7a-0b8a-4e65-97b6-cdb60265a24d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:16:53.148169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590df7a-0b8a-4e65-97b6-cdb60265a24d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:16:53.148996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590df7a-0b8a-4e65-97b6-cdb60265a24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:16:53.398630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5590df7a-0b8a-4e65-97b6-cdb60265a24d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:16:53.401475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5590df7a-0b8a-4e65-97b6-cdb60265a24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:16:53.750863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 650/1308] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 607 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:16:53.829286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ba2c1-1a45-4242-b4fd-e5a7fd0718d3 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:16:53.830198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ba2c1-1a45-4242-b4fd-e5a7fd0718d3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:16:53.831193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ba2c1-1a45-4242-b4fd-e5a7fd0718d3 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:16:53.833745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 659/1309] 10.222.0.64 () {54 vars in 965 bytes} [Wed Jul 20 08:16:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:16:53.844326 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314e989-8224-47c3-b765-4b6c61a20edd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:16:53.845212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314e989-8224-47c3-b765-4b6c61a20edd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:16:53.845970 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314e989-8224-47c3-b765-4b6c61a20edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:16:54.106214 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a314e989-8224-47c3-b765-4b6c61a20edd None None] MFA Rules not processed for user `a3b03383554346138b98becb2828851a`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:16:54.108472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a314e989-8224-47c3-b765-4b6c61a20edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:16:54.161954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:16:54.307982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 651/1310] 10.222.0.64 () {58 vars in 1062 bytes} [Wed Jul 20 08:16:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:16:54.321729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ba2c1-1a45-4242-b4fd-e5a7fd0718d3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:16:54.323786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8a5ba2c1-1a45-4242-b4fd-e5a7fd0718d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:16:54.524347 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ba2c1-1a45-4242-b4fd-e5a7fd0718d3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:16:54.705207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be6b55a-0ca2-44cc-a986-e6b8c240458a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:16:54.706079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b55a-0ca2-44cc-a986-e6b8c240458a service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:16:54.706395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b55a-0ca2-44cc-a986-e6b8c240458a service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:16:54.706718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6b55a-0ca2-44cc-a986-e6b8c240458a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:16:54.708575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0be6b55a-0ca2-44cc-a986-e6b8c240458a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:16:54.838366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:16:54.981274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6b55a-0ca2-44cc-a986-e6b8c240458a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:16:54.982807 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be6b55a-0ca2-44cc-a986-e6b8c240458a service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:16:55.030269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 660/1311] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:16:58.385548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f09af9-73dd-4499-bf99-f5b79fa5889f None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:16:58.386298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f09af9-73dd-4499-bf99-f5b79fa5889f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:16:58.386298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f09af9-73dd-4499-bf99-f5b79fa5889f None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:16:58.387896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 652/1312] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:16:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:16:58.395019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade52f75-4445-4684-8114-5476f155cf9d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:16:58.395616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade52f75-4445-4684-8114-5476f155cf9d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:16:58.395616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade52f75-4445-4684-8114-5476f155cf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:16:58.486546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ade52f75-4445-4684-8114-5476f155cf9d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:16:58.487836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ade52f75-4445-4684-8114-5476f155cf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:16:58.618858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 661/1313] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:16:58.674982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe21451-a32a-43ae-8096-2e3cbd8ba509 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:16:58.675228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe21451-a32a-43ae-8096-2e3cbd8ba509 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:16:58.675467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe21451-a32a-43ae-8096-2e3cbd8ba509 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:16:58.676860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 653/1314] 10.222.0.64 () {54 vars in 965 bytes} [Wed Jul 20 08:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:16:58.684720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b82c6-dccc-4861-a903-381323a70eb6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:16:58.684987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b82c6-dccc-4861-a903-381323a70eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:16:58.685258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b82c6-dccc-4861-a903-381323a70eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:16:58.698146 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:16:58.828789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-998b82c6-dccc-4861-a903-381323a70eb6 None None] MFA Rules not processed for user `a3b03383554346138b98becb2828851a`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:16:58.830940 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-998b82c6-dccc-4861-a903-381323a70eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:16:58.992069 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 662/1315] 10.222.0.64 () {58 vars in 1062 bytes} [Wed Jul 20 08:16:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:16:59.002676 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe21451-a32a-43ae-8096-2e3cbd8ba509 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:16:59.004343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7fe21451-a32a-43ae-8096-2e3cbd8ba509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:16:59.146272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe21451-a32a-43ae-8096-2e3cbd8ba509 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:16:59.236149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:16:59.239501 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0ef36-0a6f-4763-bcc3-14cb49ce6cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:16:59.240194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ef36-0a6f-4763-bcc3-14cb49ce6cab service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:16:59.240483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ef36-0a6f-4763-bcc3-14cb49ce6cab service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:16:59.240761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0ef36-0a6f-4763-bcc3-14cb49ce6cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:16:59.247053 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-24c0ef36-0a6f-4763-bcc3-14cb49ce6cab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:16:59.396379 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0ef36-0a6f-4763-bcc3-14cb49ce6cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:16:59.398692 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0ef36-0a6f-4763-bcc3-14cb49ce6cab service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:16:59.422833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 654/1316] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:16:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:17:02.246178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4b4b7-3282-491d-bed1-7ba33517a971 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:02.246782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4b4b7-3282-491d-bed1-7ba33517a971 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:02.246782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4b4b7-3282-491d-bed1-7ba33517a971 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:02.248182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 663/1317] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:02.254306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617123b-0534-454b-8aa7-97d150ba5bcb None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:02.254923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617123b-0534-454b-8aa7-97d150ba5bcb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:02.254923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617123b-0534-454b-8aa7-97d150ba5bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:02.331446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9617123b-0534-454b-8aa7-97d150ba5bcb None None] MFA Rules not processed for user `48574d0c8c5d4648b0a8c6d7d6d34287`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:02.332729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9617123b-0534-454b-8aa7-97d150ba5bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:02.394890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:02.448619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 655/1318] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:02.490916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4b4b7-3282-491d-bed1-7ba33517a971 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:02.492477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-87f4b4b7-3282-491d-bed1-7ba33517a971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:02.607534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4b4b7-3282-491d-bed1-7ba33517a971 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:02.610491 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:02.687108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab156f6-70ab-43c4-89eb-5d8fc0588200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:02.687935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab156f6-70ab-43c4-89eb-5d8fc0588200 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:02.688215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab156f6-70ab-43c4-89eb-5d8fc0588200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:02.688492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab156f6-70ab-43c4-89eb-5d8fc0588200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:02.690238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1ab156f6-70ab-43c4-89eb-5d8fc0588200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:02.821687 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab156f6-70ab-43c4-89eb-5d8fc0588200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:02.823274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab156f6-70ab-43c4-89eb-5d8fc0588200 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:02.845897 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 664/1319] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:17:05.708708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddec9d-8862-41f9-8d4a-338fd6246fec None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:05.709342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddec9d-8862-41f9-8d4a-338fd6246fec None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:05.709342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bddec9d-8862-41f9-8d4a-338fd6246fec None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:05.710808 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 656/1320] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:17:05.716921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ea136-ca44-4b77-aaeb-54644d786a72 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:05.717595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ea136-ca44-4b77-aaeb-54644d786a72 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:05.717595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ea136-ca44-4b77-aaeb-54644d786a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:05.766783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:05.798738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-130ea136-ca44-4b77-aaeb-54644d786a72 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:05.800037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-130ea136-ca44-4b77-aaeb-54644d786a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:05.916055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 665/1321] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:05 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:05.959024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bddec9d-8862-41f9-8d4a-338fd6246fec None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:05.960650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5bddec9d-8862-41f9-8d4a-338fd6246fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:06.146121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bddec9d-8862-41f9-8d4a-338fd6246fec None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:06.292878 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bd4d6-872b-4ecd-af35-37c5356fff2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:06.293896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bd4d6-872b-4ecd-af35-37c5356fff2f service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:06.294346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bd4d6-872b-4ecd-af35-37c5356fff2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:06.294662 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bd4d6-872b-4ecd-af35-37c5356fff2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:06.297145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b19bd4d6-872b-4ecd-af35-37c5356fff2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:06.306499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:06.445554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bd4d6-872b-4ecd-af35-37c5356fff2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:06.447288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bd4d6-872b-4ecd-af35-37c5356fff2f service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:06.469348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 657/1322] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:05 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:17:10.179979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ea99c-d332-467a-8fd8-72d20b5b9e6b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:10.181316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ea99c-d332-467a-8fd8-72d20b5b9e6b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:10.181316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085ea99c-d332-467a-8fd8-72d20b5b9e6b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:10.182944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 666/1323] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:10.189361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ced88-fd6f-40de-b656-589665c62f59 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:10.189947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ced88-fd6f-40de-b656-589665c62f59 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:10.189947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ced88-fd6f-40de-b656-589665c62f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:10.287290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9b4ced88-fd6f-40de-b656-589665c62f59 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:10.289071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9b4ced88-fd6f-40de-b656-589665c62f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:10.393942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:10.463978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 658/1324] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:10.501711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ea99c-d332-467a-8fd8-72d20b5b9e6b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:10.503477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-085ea99c-d332-467a-8fd8-72d20b5b9e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:10.653343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085ea99c-d332-467a-8fd8-72d20b5b9e6b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:10.681146 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:10.774886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e32dd-61ae-4190-abd9-5080685e2302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:10.775537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e32dd-61ae-4190-abd9-5080685e2302 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:10.775821 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e32dd-61ae-4190-abd9-5080685e2302 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:10.776103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e32dd-61ae-4190-abd9-5080685e2302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:10.777828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e44e32dd-61ae-4190-abd9-5080685e2302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:10.934273 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e32dd-61ae-4190-abd9-5080685e2302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:10.936089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e32dd-61ae-4190-abd9-5080685e2302 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:10.959581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 667/1325] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:17:13.195931 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43d015-3533-4702-9f04-5710b60549a5 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:13.196566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43d015-3533-4702-9f04-5710b60549a5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:13.196566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43d015-3533-4702-9f04-5710b60549a5 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:13.198031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 659/1326] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:13.204061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350459a-657a-4e86-b4fb-2cdd408d79e4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:13.205148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350459a-657a-4e86-b4fb-2cdd408d79e4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:13.205148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350459a-657a-4e86-b4fb-2cdd408d79e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:13.275482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:13.300697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b350459a-657a-4e86-b4fb-2cdd408d79e4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:13.302309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b350459a-657a-4e86-b4fb-2cdd408d79e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:13.428172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 668/1327] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:13.464359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43d015-3533-4702-9f04-5710b60549a5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:13.465956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7d43d015-3533-4702-9f04-5710b60549a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:13.593779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43d015-3533-4702-9f04-5710b60549a5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:13.686530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f43ba7-9151-4d85-b7e6-a21c70eea10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:13.687521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43ba7-9151-4d85-b7e6-a21c70eea10d service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:13.687810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43ba7-9151-4d85-b7e6-a21c70eea10d service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:13.688094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43ba7-9151-4d85-b7e6-a21c70eea10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:13.689896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-96f43ba7-9151-4d85-b7e6-a21c70eea10d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:13.713292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:13.837808 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f43ba7-9151-4d85-b7e6-a21c70eea10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:13.839252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f43ba7-9151-4d85-b7e6-a21c70eea10d service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:13.862002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 660/1328] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:17:16.086087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091aaaa-ce0f-4454-b6b5-2395bde86f4a None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:16.086839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091aaaa-ce0f-4454-b6b5-2395bde86f4a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:16.086839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091aaaa-ce0f-4454-b6b5-2395bde86f4a None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:16.088005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 669/1329] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:16.096126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94d7ef-718e-4a96-af8e-7c2af02f0243 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:16.096756 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94d7ef-718e-4a96-af8e-7c2af02f0243 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:16.097013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94d7ef-718e-4a96-af8e-7c2af02f0243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:16.198254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5e94d7ef-718e-4a96-af8e-7c2af02f0243 None None] MFA Rules not processed for user `48574d0c8c5d4648b0a8c6d7d6d34287`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:16.199551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5e94d7ef-718e-4a96-af8e-7c2af02f0243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:16.299794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:16.363628 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 661/1330] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:16 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:17:16.403079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091aaaa-ce0f-4454-b6b5-2395bde86f4a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:16.405366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9091aaaa-ce0f-4454-b6b5-2395bde86f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:16.545451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091aaaa-ce0f-4454-b6b5-2395bde86f4a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:16.572187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:16.640510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f153cd8-ac6c-4fbd-9fcf-bc345d619712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:16.641087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f153cd8-ac6c-4fbd-9fcf-bc345d619712 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:16.641371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f153cd8-ac6c-4fbd-9fcf-bc345d619712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:16.641652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f153cd8-ac6c-4fbd-9fcf-bc345d619712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:16.643479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7f153cd8-ac6c-4fbd-9fcf-bc345d619712 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:16.810572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f153cd8-ac6c-4fbd-9fcf-bc345d619712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:16.814862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f153cd8-ac6c-4fbd-9fcf-bc345d619712 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:16.844196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 670/1331] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:17:19.491431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a704-1ed0-4090-9e25-e46c09e41067 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:19.492028 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a704-1ed0-4090-9e25-e46c09e41067 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:19.492028 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0a704-1ed0-4090-9e25-e46c09e41067 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:19.493416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 662/1332] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:19.500387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c22911-8f4b-450e-96bc-0369b5ece06f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:19.500896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c22911-8f4b-450e-96bc-0369b5ece06f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:19.500984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c22911-8f4b-450e-96bc-0369b5ece06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:19.586706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:19.590215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-78c22911-8f4b-450e-96bc-0369b5ece06f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:19.593063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-78c22911-8f4b-450e-96bc-0369b5ece06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:19.708797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 671/1333] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:19.747089 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0a704-1ed0-4090-9e25-e46c09e41067 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:19.749068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-03c0a704-1ed0-4090-9e25-e46c09e41067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:19.867477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0a704-1ed0-4090-9e25-e46c09e41067 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:19.962103 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145fae8-4094-47c2-a11b-c89b7279bcb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:19.962904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145fae8-4094-47c2-a11b-c89b7279bcb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:19.963192 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145fae8-4094-47c2-a11b-c89b7279bcb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:19.963473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145fae8-4094-47c2-a11b-c89b7279bcb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:19.965206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9145fae8-4094-47c2-a11b-c89b7279bcb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:19.996547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:20.097912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145fae8-4094-47c2-a11b-c89b7279bcb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:20.099419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145fae8-4094-47c2-a11b-c89b7279bcb4 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:20.119611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 663/1334] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:17:22.308689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429d99-3294-45c1-8106-fd9255bc5175 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:22.309272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429d99-3294-45c1-8106-fd9255bc5175 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:22.309272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429d99-3294-45c1-8106-fd9255bc5175 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:22.311372 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 672/1335] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:22.317325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2082cd-dc02-4f1d-bb81-efcff6adb6b9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:22.317979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2082cd-dc02-4f1d-bb81-efcff6adb6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:22.317979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2082cd-dc02-4f1d-bb81-efcff6adb6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:22.387915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8e2082cd-dc02-4f1d-bb81-efcff6adb6b9 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:22.389843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8e2082cd-dc02-4f1d-bb81-efcff6adb6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:22.470542 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:22.494897 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 664/1336] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:22.548770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90429d99-3294-45c1-8106-fd9255bc5175 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:22.550556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-90429d99-3294-45c1-8106-fd9255bc5175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:22.683271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90429d99-3294-45c1-8106-fd9255bc5175 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:22.728243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:22.787935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1559b78f-3a00-497e-be87-6f31de2703a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:22.788594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559b78f-3a00-497e-be87-6f31de2703a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:22.788870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559b78f-3a00-497e-be87-6f31de2703a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:22.789144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1559b78f-3a00-497e-be87-6f31de2703a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:22.790859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1559b78f-3a00-497e-be87-6f31de2703a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:22.917272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559b78f-3a00-497e-be87-6f31de2703a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:22.918783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1559b78f-3a00-497e-be87-6f31de2703a2 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:22.942416 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 673/1337] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:17:25.495847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ccb7a-7942-44c8-bfca-ddbba6ef5c42 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:25.496391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ccb7a-7942-44c8-bfca-ddbba6ef5c42 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:25.496391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ccb7a-7942-44c8-bfca-ddbba6ef5c42 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:25.497817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 665/1338] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:25.503634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1033b72-b9b9-411b-8860-0d59b1d8e4e4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:25.504155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1033b72-b9b9-411b-8860-0d59b1d8e4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:25.504440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1033b72-b9b9-411b-8860-0d59b1d8e4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:25.581588 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e1033b72-b9b9-411b-8860-0d59b1d8e4e4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:25.582881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e1033b72-b9b9-411b-8860-0d59b1d8e4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:25.592947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:25.715348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 674/1339] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:25.754726 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ccb7a-7942-44c8-bfca-ddbba6ef5c42 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:25.756424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f76ccb7a-7942-44c8-bfca-ddbba6ef5c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:25.937054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ccb7a-7942-44c8-bfca-ddbba6ef5c42 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:26.022101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db94cd-85de-4db9-969d-023f5016a28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:26.022664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db94cd-85de-4db9-969d-023f5016a28a service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:26.022949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db94cd-85de-4db9-969d-023f5016a28a service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:26.023227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db94cd-85de-4db9-969d-023f5016a28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:26.024998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c5db94cd-85de-4db9-969d-023f5016a28a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:26.072204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:26.157972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db94cd-85de-4db9-969d-023f5016a28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:26.159482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db94cd-85de-4db9-969d-023f5016a28a service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:26.180615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 666/1340] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:17:30.601070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd2310-e1b9-48d8-964e-9a4c39b5e962 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:30.601958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd2310-e1b9-48d8-964e-9a4c39b5e962 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:30.602125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd2310-e1b9-48d8-964e-9a4c39b5e962 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:30.604633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 675/1341] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:30.610784 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593185f-a770-403b-9be0-bb1732a6e4d1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:30.611340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593185f-a770-403b-9be0-bb1732a6e4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:30.611340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593185f-a770-403b-9be0-bb1732a6e4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:30.714539 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c593185f-a770-403b-9be0-bb1732a6e4d1 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:30.716839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c593185f-a770-403b-9be0-bb1732a6e4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:30.822429 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:30.836678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 667/1342] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:17:30.880108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd2310-e1b9-48d8-964e-9a4c39b5e962 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:30.881670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c4bd2310-e1b9-48d8-964e-9a4c39b5e962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:31.007691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd2310-e1b9-48d8-964e-9a4c39b5e962 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:31.053776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:31.095474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad8fdb-7dde-4f00-a286-0d9b269eee11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:31.096041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad8fdb-7dde-4f00-a286-0d9b269eee11 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:31.096323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad8fdb-7dde-4f00-a286-0d9b269eee11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:31.096597 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad8fdb-7dde-4f00-a286-0d9b269eee11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:31.098282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b0ad8fdb-7dde-4f00-a286-0d9b269eee11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:31.220743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad8fdb-7dde-4f00-a286-0d9b269eee11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:31.222399 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad8fdb-7dde-4f00-a286-0d9b269eee11 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:31.243889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 676/1343] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:17:34.352408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef247b78-2530-405d-bf89-9bcc7aadd7f6 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:34.353058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef247b78-2530-405d-bf89-9bcc7aadd7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:34.353058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef247b78-2530-405d-bf89-9bcc7aadd7f6 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:34.354342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 668/1344] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:17:34.362526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354638b-d0f4-4c27-b481-241e73e4e261 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:34.363172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354638b-d0f4-4c27-b481-241e73e4e261 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:34.363600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354638b-d0f4-4c27-b481-241e73e4e261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:34.474404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b354638b-d0f4-4c27-b481-241e73e4e261 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:34.475707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b354638b-d0f4-4c27-b481-241e73e4e261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:34.505788 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:34.612735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 677/1345] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:34.655160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef247b78-2530-405d-bf89-9bcc7aadd7f6 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:34.658221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ef247b78-2530-405d-bf89-9bcc7aadd7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:34.835084 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef247b78-2530-405d-bf89-9bcc7aadd7f6 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:34.962249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ad707-0ef6-4ae1-a821-c2956c79f976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:34.963296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ad707-0ef6-4ae1-a821-c2956c79f976 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:34.963798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ad707-0ef6-4ae1-a821-c2956c79f976 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:34.964293 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ad707-0ef6-4ae1-a821-c2956c79f976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:34.968678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d47ad707-0ef6-4ae1-a821-c2956c79f976 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:35.041924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:35.127651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ad707-0ef6-4ae1-a821-c2956c79f976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:35.129138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47ad707-0ef6-4ae1-a821-c2956c79f976 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:35.152631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 669/1346] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:17:40.279627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25969243-a499-47c0-b5c0-dcf9e5dc4fee None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:40.280152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25969243-a499-47c0-b5c0-dcf9e5dc4fee None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:40.280280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25969243-a499-47c0-b5c0-dcf9e5dc4fee None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:40.282280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 678/1347] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:40.290055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7f27d-6e7f-42fb-bd78-6e1ee7f79a8f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:40.290807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7f27d-6e7f-42fb-bd78-6e1ee7f79a8f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:40.290916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7f27d-6e7f-42fb-bd78-6e1ee7f79a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:40.387441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-72a7f27d-6e7f-42fb-bd78-6e1ee7f79a8f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:40.388716 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-72a7f27d-6e7f-42fb-bd78-6e1ee7f79a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:40.525758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:40.530015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 670/1348] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:40.578142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25969243-a499-47c0-b5c0-dcf9e5dc4fee None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:40.579760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-25969243-a499-47c0-b5c0-dcf9e5dc4fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:40.747640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25969243-a499-47c0-b5c0-dcf9e5dc4fee None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:40.827599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:40.861221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3e085-c831-4975-b2b2-a2ac6efbe745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:40.861848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3e085-c831-4975-b2b2-a2ac6efbe745 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:40.862136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3e085-c831-4975-b2b2-a2ac6efbe745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:40.862419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3e085-c831-4975-b2b2-a2ac6efbe745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:40.864238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3ac3e085-c831-4975-b2b2-a2ac6efbe745 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:41.013908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3e085-c831-4975-b2b2-a2ac6efbe745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:41.015426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3e085-c831-4975-b2b2-a2ac6efbe745 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:41.035270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 679/1349] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:17:46.409811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fe6d8-db7d-49f6-9292-99620ca0e3a9 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:46.410772 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fe6d8-db7d-49f6-9292-99620ca0e3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:46.410772 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23fe6d8-db7d-49f6-9292-99620ca0e3a9 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:46.412216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 671/1350] 10.222.0.64 () {54 vars in 921 bytes} [Wed Jul 20 08:17:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:46.422478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7229f2-c7b8-4df7-95fb-72d1aac9e2d5 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:46.426459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7229f2-c7b8-4df7-95fb-72d1aac9e2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:46.426459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7229f2-c7b8-4df7-95fb-72d1aac9e2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:46.613837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bc7229f2-c7b8-4df7-95fb-72d1aac9e2d5 None None] MFA Rules not processed for user `928054a577d14522bd0f00417f4f8c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:46.615711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bc7229f2-c7b8-4df7-95fb-72d1aac9e2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:46.651436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:46.842661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 680/1351] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Jul 20 08:17:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:51.857483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fe11f-61d3-45c3-b848-18f746178a28 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:51.858308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fe11f-61d3-45c3-b848-18f746178a28 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:51.858308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fe11f-61d3-45c3-b848-18f746178a28 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:51.859761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 672/1352] 10.222.0.64 () {54 vars in 921 bytes} [Wed Jul 20 08:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:51.865343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230862b3-f8b9-4867-88a9-e9590181c635 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:51.866145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230862b3-f8b9-4867-88a9-e9590181c635 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:51.866145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230862b3-f8b9-4867-88a9-e9590181c635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:51.961363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-230862b3-f8b9-4867-88a9-e9590181c635 None None] MFA Rules not processed for user `928054a577d14522bd0f00417f4f8c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:51.962493 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-230862b3-f8b9-4867-88a9-e9590181c635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:52.076193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:52.128936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 681/1353] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Jul 20 08:17:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:17:53.935325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd741b6-e46b-491e-8a52-972a68e91fb7 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:53.935905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd741b6-e46b-491e-8a52-972a68e91fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:53.935905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd741b6-e46b-491e-8a52-972a68e91fb7 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:53.937423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 673/1354] 10.222.0.64 () {54 vars in 921 bytes} [Wed Jul 20 08:17:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:53.944610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c28bd-8bb3-4d21-9bf5-058764a1f06d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:53.945420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c28bd-8bb3-4d21-9bf5-058764a1f06d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:53.945646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c28bd-8bb3-4d21-9bf5-058764a1f06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:54.055255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e63c28bd-8bb3-4d21-9bf5-058764a1f06d None None] MFA Rules not processed for user `928054a577d14522bd0f00417f4f8c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:54.056564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e63c28bd-8bb3-4d21-9bf5-058764a1f06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:54.203187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 682/1355] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Jul 20 08:17:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:17:58.220769 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d8e90-0f1e-47c6-8a36-5ff6fdfc128e None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:58.221811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d8e90-0f1e-47c6-8a36-5ff6fdfc128e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:58.222291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d8e90-0f1e-47c6-8a36-5ff6fdfc128e None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:58.223713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 674/1356] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:17:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:17:58.229458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d9cdc-a329-4c55-ad5f-7c83bbe3f846 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:58.230761 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d9cdc-a329-4c55-ad5f-7c83bbe3f846 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:58.230917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d9cdc-a329-4c55-ad5f-7c83bbe3f846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:58.290953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:58.350360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-650d9cdc-a329-4c55-ad5f-7c83bbe3f846 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:58.352287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-650d9cdc-a329-4c55-ad5f-7c83bbe3f846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:58.506687 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 683/1357] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:58.583940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb638f9-5cfb-4930-9e3a-fc574d0aa846 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:58.584972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb638f9-5cfb-4930-9e3a-fc574d0aa846 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:58.585432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb638f9-5cfb-4930-9e3a-fc574d0aa846 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:58.587608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 675/1358] 10.222.0.64 () {54 vars in 962 bytes} [Wed Jul 20 08:17:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:17:58.602183 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a6f20-b85e-4ab7-be34-2446a6c05e17 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:58.602738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a6f20-b85e-4ab7-be34-2446a6c05e17 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:58.603018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a6f20-b85e-4ab7-be34-2446a6c05e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:58.712261 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2e0a6f20-b85e-4ab7-be34-2446a6c05e17 None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:17:58.714814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2e0a6f20-b85e-4ab7-be34-2446a6c05e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:58.796460 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:58.892030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 684/1359] 10.222.0.64 () {58 vars in 1059 bytes} [Wed Jul 20 08:17:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:17:58.902512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb638f9-5cfb-4930-9e3a-fc574d0aa846 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:17:58.905616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ddb638f9-5cfb-4930-9e3a-fc574d0aa846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:59.154384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb638f9-5cfb-4930-9e3a-fc574d0aa846 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:17:59.246049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:17:59.291548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a203677-0e3a-4c37-9087-e33c56587bd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:17:59.292757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a203677-0e3a-4c37-9087-e33c56587bd3 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:17:59.293066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a203677-0e3a-4c37-9087-e33c56587bd3 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:17:59.293521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a203677-0e3a-4c37-9087-e33c56587bd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:17:59.295307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8a203677-0e3a-4c37-9087-e33c56587bd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:17:59.403176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a203677-0e3a-4c37-9087-e33c56587bd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:17:59.406290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a203677-0e3a-4c37-9087-e33c56587bd3 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:17:59.435465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 676/1360] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:17:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:18:02.434890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d2953-244a-4f01-9d27-5712543862fa None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:02.435448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d2953-244a-4f01-9d27-5712543862fa None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:02.435448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d2953-244a-4f01-9d27-5712543862fa None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:02.436863 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 685/1361] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:02.442455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e28809-2fc7-4a45-863e-fedac1621a7c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:02.443025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e28809-2fc7-4a45-863e-fedac1621a7c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:02.443115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e28809-2fc7-4a45-863e-fedac1621a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:02.549353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-45e28809-2fc7-4a45-863e-fedac1621a7c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:02.550642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-45e28809-2fc7-4a45-863e-fedac1621a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:02.610647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:02.700043 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 677/1362] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:02.749984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d2953-244a-4f01-9d27-5712543862fa None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:02.749984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-874d2953-244a-4f01-9d27-5712543862fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:02.970009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d2953-244a-4f01-9d27-5712543862fa None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:02.986354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:03.154631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f1d5e-c039-4a7d-a042-79e2a2027a3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:03.155327 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f1d5e-c039-4a7d-a042-79e2a2027a3e service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:03.155610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f1d5e-c039-4a7d-a042-79e2a2027a3e service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:03.155888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f1d5e-c039-4a7d-a042-79e2a2027a3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:03.157626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5b2f1d5e-c039-4a7d-a042-79e2a2027a3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:03.273690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f1d5e-c039-4a7d-a042-79e2a2027a3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:03.275547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f1d5e-c039-4a7d-a042-79e2a2027a3e service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:03.298316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 686/1363] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:05.764797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7796b-93de-4ae5-8d1b-8e29ea11a4da None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:05.765367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7796b-93de-4ae5-8d1b-8e29ea11a4da None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:05.765367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7796b-93de-4ae5-8d1b-8e29ea11a4da None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:05.766864 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 678/1364] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:05.772894 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be57841-8719-40ca-8917-301be91ffd63 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:05.773438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be57841-8719-40ca-8917-301be91ffd63 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:05.773523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be57841-8719-40ca-8917-301be91ffd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:05.795243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:05.865722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8be57841-8719-40ca-8917-301be91ffd63 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:05.866999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8be57841-8719-40ca-8917-301be91ffd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:05.999964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 687/1365] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:06.050460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e683f-4b51-4a75-8717-3066b931c72b None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:06.050921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e683f-4b51-4a75-8717-3066b931c72b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:06.051305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e683f-4b51-4a75-8717-3066b931c72b None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:06.053131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 679/1366] 10.222.0.64 () {54 vars in 962 bytes} [Wed Jul 20 08:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:06.060205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c8d97-161b-4a07-a314-a9e0f5c03370 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:06.060565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c8d97-161b-4a07-a314-a9e0f5c03370 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:06.060765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c8d97-161b-4a07-a314-a9e0f5c03370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:06.169403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bd4c8d97-161b-4a07-a314-a9e0f5c03370 None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:06.171335 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bd4c8d97-161b-4a07-a314-a9e0f5c03370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:06.222794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:06.340817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 688/1367] 10.222.0.64 () {58 vars in 1059 bytes} [Wed Jul 20 08:18:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:06.350178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e683f-4b51-4a75-8717-3066b931c72b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:06.351962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-681e683f-4b51-4a75-8717-3066b931c72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:06.540314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e683f-4b51-4a75-8717-3066b931c72b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:06.572843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:06.733581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32678cce-1d76-44a8-a498-5198814688cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:06.734561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32678cce-1d76-44a8-a498-5198814688cc service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:06.734995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32678cce-1d76-44a8-a498-5198814688cc service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:06.735370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32678cce-1d76-44a8-a498-5198814688cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:06.737159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-32678cce-1d76-44a8-a498-5198814688cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:06.849586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32678cce-1d76-44a8-a498-5198814688cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:06.852081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32678cce-1d76-44a8-a498-5198814688cc service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:06.876185 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 680/1368] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:07.048408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c0de7-c85d-492c-9c44-7ea220baf66b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:07.048976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c0de7-c85d-492c-9c44-7ea220baf66b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:07.049459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c0de7-c85d-492c-9c44-7ea220baf66b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:07.051385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 689/1369] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:18:07.057933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32a88f-1bab-416c-bb21-b6da6c456f89 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:07.058437 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32a88f-1bab-416c-bb21-b6da6c456f89 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:07.059419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32a88f-1bab-416c-bb21-b6da6c456f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:07.099104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:07.154601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8f32a88f-1bab-416c-bb21-b6da6c456f89 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:07.155844 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8f32a88f-1bab-416c-bb21-b6da6c456f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:07.271203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 681/1370] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:07.600488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f63c3-1e43-452c-aaee-97b14bd2de81 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:07.600785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f63c3-1e43-452c-aaee-97b14bd2de81 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:07.601055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f63c3-1e43-452c-aaee-97b14bd2de81 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:07.602611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 690/1371] 10.222.0.64 () {54 vars in 964 bytes} [Wed Jul 20 08:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:07.607656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163d2ef-83f0-4bb0-b422-27c8a37ee33f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:07.608001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163d2ef-83f0-4bb0-b422-27c8a37ee33f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:07.608264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163d2ef-83f0-4bb0-b422-27c8a37ee33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:07.695488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8163d2ef-83f0-4bb0-b422-27c8a37ee33f None None] MFA Rules not processed for user `41a647bd72f344fa963dcd3eb5434b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:07.696733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8163d2ef-83f0-4bb0-b422-27c8a37ee33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:07.754918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:07.792937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 682/1372] 10.222.0.64 () {58 vars in 1061 bytes} [Wed Jul 20 08:18:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:07.801137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f63c3-1e43-452c-aaee-97b14bd2de81 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:07.803780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0f8f63c3-1e43-452c-aaee-97b14bd2de81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:07.946395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f63c3-1e43-452c-aaee-97b14bd2de81 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:07.997978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:08.045069 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce3e8-d41c-49f2-97ad-718a840eec9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:08.045747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce3e8-d41c-49f2-97ad-718a840eec9c service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:08.046071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce3e8-d41c-49f2-97ad-718a840eec9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:08.046527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce3e8-d41c-49f2-97ad-718a840eec9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:08.048575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-841ce3e8-d41c-49f2-97ad-718a840eec9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:08.177010 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ce3e8-d41c-49f2-97ad-718a840eec9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:08.180246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ce3e8-d41c-49f2-97ad-718a840eec9c service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:08.210989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 691/1373] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:09.110229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666409a3-c679-4dd9-b0b7-afb86654a168 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:09.110818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666409a3-c679-4dd9-b0b7-afb86654a168 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:09.111107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666409a3-c679-4dd9-b0b7-afb86654a168 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:09.112621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 683/1374] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:09.118275 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca753de-2fdf-4aac-a95a-d0923c3d0c68 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:09.118777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca753de-2fdf-4aac-a95a-d0923c3d0c68 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:09.118888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca753de-2fdf-4aac-a95a-d0923c3d0c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:09.188715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8ca753de-2fdf-4aac-a95a-d0923c3d0c68 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:09.189876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8ca753de-2fdf-4aac-a95a-d0923c3d0c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:09.213923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:09.296418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 692/1375] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:09.341984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666409a3-c679-4dd9-b0b7-afb86654a168 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:09.343604 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-666409a3-c679-4dd9-b0b7-afb86654a168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:09.507472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666409a3-c679-4dd9-b0b7-afb86654a168 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:09.564146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:09.632696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197c0bf-dd19-4f38-a4c8-0f296d0b9faa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:09.633354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197c0bf-dd19-4f38-a4c8-0f296d0b9faa service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:09.633636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197c0bf-dd19-4f38-a4c8-0f296d0b9faa service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:09.633910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197c0bf-dd19-4f38-a4c8-0f296d0b9faa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:09.635853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b197c0bf-dd19-4f38-a4c8-0f296d0b9faa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:09.746502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197c0bf-dd19-4f38-a4c8-0f296d0b9faa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:09.748297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197c0bf-dd19-4f38-a4c8-0f296d0b9faa service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:09.770973 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 684/1376] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:11.955787 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf0e8c-5b64-4346-8908-6883ebe44241 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:11.956307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf0e8c-5b64-4346-8908-6883ebe44241 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:11.956307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf0e8c-5b64-4346-8908-6883ebe44241 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:11.957878 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 693/1377] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:11.965391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126f96f-0e32-4e01-b8d2-d62918e8786d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:11.966083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126f96f-0e32-4e01-b8d2-d62918e8786d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:11.966083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126f96f-0e32-4e01-b8d2-d62918e8786d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:12.056461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e126f96f-0e32-4e01-b8d2-d62918e8786d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:12.057884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e126f96f-0e32-4e01-b8d2-d62918e8786d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:12.066965 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:12.201467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 685/1378] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:12.258295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf0e8c-5b64-4346-8908-6883ebe44241 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:12.260999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1fdf0e8c-5b64-4346-8908-6883ebe44241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:12.481192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf0e8c-5b64-4346-8908-6883ebe44241 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:12.496455 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:12.642155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e772ade-985b-4dc3-ac27-30a8b2884d5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:12.642864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e772ade-985b-4dc3-ac27-30a8b2884d5f service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:12.643233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e772ade-985b-4dc3-ac27-30a8b2884d5f service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:12.643404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e772ade-985b-4dc3-ac27-30a8b2884d5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:12.645194 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5e772ade-985b-4dc3-ac27-30a8b2884d5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:12.777512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e772ade-985b-4dc3-ac27-30a8b2884d5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:12.779257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e772ade-985b-4dc3-ac27-30a8b2884d5f service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:12.800494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 694/1379] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:14.983220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff6f9d-e065-4a11-bf38-0efa70afe699 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:14.983782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff6f9d-e065-4a11-bf38-0efa70afe699 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:14.983909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ff6f9d-e065-4a11-bf38-0efa70afe699 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:14.985437 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 686/1380] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:18:14.991016 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a7838-c54c-4a39-9361-f577887575f7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:14.991612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a7838-c54c-4a39-9361-f577887575f7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:14.991612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a7838-c54c-4a39-9361-f577887575f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:15.003122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:15.104393 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-377a7838-c54c-4a39-9361-f577887575f7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:15.106328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-377a7838-c54c-4a39-9361-f577887575f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:15.293411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 695/1381] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:15.374052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff6f9d-e065-4a11-bf38-0efa70afe699 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:15.376232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-39ff6f9d-e065-4a11-bf38-0efa70afe699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:15.558189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:15.571443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ff6f9d-e065-4a11-bf38-0efa70afe699 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:15.717359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d78e7f-a078-430e-a310-be1d05d29398 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:15.718595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78e7f-a078-430e-a310-be1d05d29398 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:15.718595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78e7f-a078-430e-a310-be1d05d29398 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:15.718595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d78e7f-a078-430e-a310-be1d05d29398 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:15.720323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-27d78e7f-a078-430e-a310-be1d05d29398 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:15.835673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d78e7f-a078-430e-a310-be1d05d29398 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:15.837228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d78e7f-a078-430e-a310-be1d05d29398 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:15.849067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:15.859619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 687/1382] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:18.165722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51670619-36e7-4b73-97b0-70f52e827e65 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:18.166375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51670619-36e7-4b73-97b0-70f52e827e65 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:18.166375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51670619-36e7-4b73-97b0-70f52e827e65 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:18.167830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 696/1383] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:18:18.173421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b8c8c-1f1c-43fa-8075-aad33a5e11b7 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:18.173952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b8c8c-1f1c-43fa-8075-aad33a5e11b7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:18.174110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b8c8c-1f1c-43fa-8075-aad33a5e11b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:18.250472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-851b8c8c-1f1c-43fa-8075-aad33a5e11b7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:18.251764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-851b8c8c-1f1c-43fa-8075-aad33a5e11b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:18.384243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 688/1384] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:18.433463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51670619-36e7-4b73-97b0-70f52e827e65 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:18.435207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-51670619-36e7-4b73-97b0-70f52e827e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:18.550414 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:18.630952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51670619-36e7-4b73-97b0-70f52e827e65 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:18.764608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b43f56-6752-4da7-85ba-bb64705221d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:18.765158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b43f56-6752-4da7-85ba-bb64705221d2 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:18.765442 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b43f56-6752-4da7-85ba-bb64705221d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:18.765722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b43f56-6752-4da7-85ba-bb64705221d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:18.767552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-50b43f56-6752-4da7-85ba-bb64705221d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:18.840413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:18.908080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b43f56-6752-4da7-85ba-bb64705221d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:18.909710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b43f56-6752-4da7-85ba-bb64705221d2 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:18.943020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 697/1385] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:21.604567 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b432-a45c-4dea-9147-7f693e795f8f None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:21.605201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b432-a45c-4dea-9147-7f693e795f8f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:21.605201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b432-a45c-4dea-9147-7f693e795f8f None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:21.606929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 689/1386] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:21.613206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e1887-591d-4361-8402-b040d08e6f57 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:21.613759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e1887-591d-4361-8402-b040d08e6f57 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:21.613856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e1887-591d-4361-8402-b040d08e6f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:21.704398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-692e1887-591d-4361-8402-b040d08e6f57 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:21.705898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-692e1887-591d-4361-8402-b040d08e6f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:21.856083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 698/1387] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:21.904317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453b432-a45c-4dea-9147-7f693e795f8f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:21.905859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e453b432-a45c-4dea-9147-7f693e795f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:21.993500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:22.078188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453b432-a45c-4dea-9147-7f693e795f8f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:22.214162 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21537c8-296f-4c79-88b5-5314219ad8a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:22.214790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21537c8-296f-4c79-88b5-5314219ad8a4 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:22.215082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21537c8-296f-4c79-88b5-5314219ad8a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:22.215362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21537c8-296f-4c79-88b5-5314219ad8a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:22.217073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c21537c8-296f-4c79-88b5-5314219ad8a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:22.261515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:22.323441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21537c8-296f-4c79-88b5-5314219ad8a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:22.325435 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21537c8-296f-4c79-88b5-5314219ad8a4 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:22.350623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 690/1388] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:23.923681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b819c-cc92-4a8f-b9c8-8433a04091a0 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:23.924333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b819c-cc92-4a8f-b9c8-8433a04091a0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:23.924333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b819c-cc92-4a8f-b9c8-8433a04091a0 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:23.925657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 699/1389] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:23.931972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802720f-90a7-4372-8ca8-aba2824aa24e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:23.932705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802720f-90a7-4372-8ca8-aba2824aa24e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:23.932705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802720f-90a7-4372-8ca8-aba2824aa24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:24.005869 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0802720f-90a7-4372-8ca8-aba2824aa24e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:24.007283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0802720f-90a7-4372-8ca8-aba2824aa24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:24.115348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 691/1390] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:18:24.176187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e5719-2600-407d-8702-be65495c9060 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:24.176416 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e5719-2600-407d-8702-be65495c9060 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:24.176944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3e5719-2600-407d-8702-be65495c9060 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:24.178483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 700/1391] 10.222.0.64 () {54 vars in 966 bytes} [Wed Jul 20 08:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:24.184457 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08450b-f105-4fbc-91e0-0947b9c7c7b3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:24.184731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08450b-f105-4fbc-91e0-0947b9c7c7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:24.185077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08450b-f105-4fbc-91e0-0947b9c7c7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:24.206272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:24.274358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2b08450b-f105-4fbc-91e0-0947b9c7c7b3 None None] MFA Rules not processed for user `3778997232d44cd3a3c4fb3d53e4db53`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:24.275711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2b08450b-f105-4fbc-91e0-0947b9c7c7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:24.373343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 692/1392] 10.222.0.64 () {58 vars in 1063 bytes} [Wed Jul 20 08:18:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:24.381443 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e5719-2600-407d-8702-be65495c9060 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:24.382949 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5e3e5719-2600-407d-8702-be65495c9060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:24.422077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:24.509556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3e5719-2600-407d-8702-be65495c9060 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:24.518915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b54d4-a9b3-472d-a130-eb7dfaef35ee None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:24.519446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b54d4-a9b3-472d-a130-eb7dfaef35ee None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:24.519972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b54d4-a9b3-472d-a130-eb7dfaef35ee None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:24.521434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 693/1393] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:18:24.529305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272f29e-fed9-4bd1-b86d-eda5007aa33b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:24.529656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272f29e-fed9-4bd1-b86d-eda5007aa33b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:24.529967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272f29e-fed9-4bd1-b86d-eda5007aa33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:24.596872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a49da9-a11a-4b8b-b575-c86e92ea891e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3778997232d44cd3a3c4fb3d53e4db53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:24.597572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a49da9-a11a-4b8b-b575-c86e92ea891e service glance] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:24.597799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a49da9-a11a-4b8b-b575-c86e92ea891e service glance] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:24.598187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a49da9-a11a-4b8b-b575-c86e92ea891e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:24.599823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c2a49da9-a11a-4b8b-b575-c86e92ea891e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:24.608631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a272f29e-fed9-4bd1-b86d-eda5007aa33b None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:24.609970 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a272f29e-fed9-4bd1-b86d-eda5007aa33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:24.675911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:24.687855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:24.727336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a49da9-a11a-4b8b-b575-c86e92ea891e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:24.729595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a49da9-a11a-4b8b-b575-c86e92ea891e service glance] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:24.731170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 694/1394] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:18:24.750673 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 701/1395] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:24.777954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272f29e-fed9-4bd1-b86d-eda5007aa33b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:24.779810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a272f29e-fed9-4bd1-b86d-eda5007aa33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:24.905592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272f29e-fed9-4bd1-b86d-eda5007aa33b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:24.985249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:25.044620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b2e25-57fc-49de-a549-c8e8b394bf13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:25.045286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b2e25-57fc-49de-a549-c8e8b394bf13 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:25.045576 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b2e25-57fc-49de-a549-c8e8b394bf13 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:25.045849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b2e25-57fc-49de-a549-c8e8b394bf13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:25.047595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e75b2e25-57fc-49de-a549-c8e8b394bf13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:25.154255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b2e25-57fc-49de-a549-c8e8b394bf13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:25.155992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b2e25-57fc-49de-a549-c8e8b394bf13 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:25.176092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 695/1396] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:18:25.493771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd1a17-b9e8-46be-9433-d57e348e56b2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:25.495402 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd1a17-b9e8-46be-9433-d57e348e56b2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:25.495754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd1a17-b9e8-46be-9433-d57e348e56b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:25.576928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ebfd1a17-b9e8-46be-9433-d57e348e56b2 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:25.578089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ebfd1a17-b9e8-46be-9433-d57e348e56b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:25.672547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 702/1397] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:18:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:25.732019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57828853-39ec-46e8-96ad-60f1687fedbf None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:25.732346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57828853-39ec-46e8-96ad-60f1687fedbf None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:25.732660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57828853-39ec-46e8-96ad-60f1687fedbf None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:25.734621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 696/1398] 10.222.0.64 () {54 vars in 943 bytes} [Wed Jul 20 08:18:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:25.741503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99261071-6e56-4b64-842d-ba871a1850a7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:25.741873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99261071-6e56-4b64-842d-ba871a1850a7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:25.742304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99261071-6e56-4b64-842d-ba871a1850a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:25.804313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:25.826496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-99261071-6e56-4b64-842d-ba871a1850a7 None None] MFA Rules not processed for user `c3cf87f0515b460dad8c0ff5591c2685`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:25.827824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-99261071-6e56-4b64-842d-ba871a1850a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:25.958222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 703/1399] 10.222.0.64 () {58 vars in 1040 bytes} [Wed Jul 20 08:18:25 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:18:25.968558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57828853-39ec-46e8-96ad-60f1687fedbf None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:25.970562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-57828853-39ec-46e8-96ad-60f1687fedbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:26.065747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:26.131890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57828853-39ec-46e8-96ad-60f1687fedbf None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:26.251283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a74583-c445-4768-8715-5458b6d66a4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:26.251942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a74583-c445-4768-8715-5458b6d66a4a service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:26.252229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a74583-c445-4768-8715-5458b6d66a4a service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:26.252508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a74583-c445-4768-8715-5458b6d66a4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:26.254248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-37a74583-c445-4768-8715-5458b6d66a4a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:26.328796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:26.352303 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a74583-c445-4768-8715-5458b6d66a4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:26.354341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a74583-c445-4768-8715-5458b6d66a4a service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:26.367127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 697/1400] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:18:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:18:27.384216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9931324-f2b6-4315-ad4d-2531252d9215 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:27.384883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9931324-f2b6-4315-ad4d-2531252d9215 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:27.384980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9931324-f2b6-4315-ad4d-2531252d9215 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:27.386959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 704/1401] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:27.392285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a6b5-ee6b-46ec-ac3c-341fb0752f41 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:27.392785 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a6b5-ee6b-46ec-ac3c-341fb0752f41 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:27.392878 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64a6b5-ee6b-46ec-ac3c-341fb0752f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:27.489352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-fb64a6b5-ee6b-46ec-ac3c-341fb0752f41 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:27.491697 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fb64a6b5-ee6b-46ec-ac3c-341fb0752f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:27.634903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 698/1402] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:18:27.676653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9931324-f2b6-4315-ad4d-2531252d9215 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:27.678312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c9931324-f2b6-4315-ad4d-2531252d9215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:27.820037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:27.886974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9931324-f2b6-4315-ad4d-2531252d9215 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:28.064544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92743bd5-de54-4fbe-b892-87a210f64cc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:28.065372 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92743bd5-de54-4fbe-b892-87a210f64cc9 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:28.065667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92743bd5-de54-4fbe-b892-87a210f64cc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:28.065948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92743bd5-de54-4fbe-b892-87a210f64cc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:28.067716 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-92743bd5-de54-4fbe-b892-87a210f64cc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:28.153359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:28.185814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92743bd5-de54-4fbe-b892-87a210f64cc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:28.187966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92743bd5-de54-4fbe-b892-87a210f64cc9 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:28.214502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 705/1403] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:30.460777 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd56df-4313-4c3a-a807-974737c7bfd0 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:30.461333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd56df-4313-4c3a-a807-974737c7bfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:30.461333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd56df-4313-4c3a-a807-974737c7bfd0 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:30.467037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 699/1404] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:30.468521 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dced147-7b9a-4063-9049-133ff46360a9 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:30.468960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dced147-7b9a-4063-9049-133ff46360a9 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:30.469137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dced147-7b9a-4063-9049-133ff46360a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:30.545292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3dced147-7b9a-4063-9049-133ff46360a9 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:30.546543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3dced147-7b9a-4063-9049-133ff46360a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:30.674777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 706/1405] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:18:30.716854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd56df-4313-4c3a-a807-974737c7bfd0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:30.718279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-73fd56df-4313-4c3a-a807-974737c7bfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:30.788976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:30.920914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd56df-4313-4c3a-a807-974737c7bfd0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:31.066342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bff9d-6397-4fc2-b9ef-0eacc6ee67fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:31.067345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bff9d-6397-4fc2-b9ef-0eacc6ee67fa service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:31.067793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bff9d-6397-4fc2-b9ef-0eacc6ee67fa service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:31.068506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bff9d-6397-4fc2-b9ef-0eacc6ee67fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:31.070444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dd7bff9d-6397-4fc2-b9ef-0eacc6ee67fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:31.092521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:31.180914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bff9d-6397-4fc2-b9ef-0eacc6ee67fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:31.183805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7bff9d-6397-4fc2-b9ef-0eacc6ee67fa service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:31.214823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 700/1406] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:33.404618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319d6d1-bd8c-49d0-a81b-c97c36d4a440 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:33.405182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319d6d1-bd8c-49d0-a81b-c97c36d4a440 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:33.405182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319d6d1-bd8c-49d0-a81b-c97c36d4a440 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:33.408266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 707/1407] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:18:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:33.412508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305375-b581-4ccb-b477-3fdd449c19b3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:33.413255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305375-b581-4ccb-b477-3fdd449c19b3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:33.413255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36305375-b581-4ccb-b477-3fdd449c19b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:33.486311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-36305375-b581-4ccb-b477-3fdd449c19b3 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:33.487595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-36305375-b581-4ccb-b477-3fdd449c19b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:33.601492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:33.609666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 701/1408] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:33.659320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319d6d1-bd8c-49d0-a81b-c97c36d4a440 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:33.660797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7319d6d1-bd8c-49d0-a81b-c97c36d4a440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:33.716250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:33.837953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319d6d1-bd8c-49d0-a81b-c97c36d4a440 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:33.989530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8031af5-6858-4354-94eb-70ed18faaaf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:33.990605 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8031af5-6858-4354-94eb-70ed18faaaf7 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:33.991145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8031af5-6858-4354-94eb-70ed18faaaf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:33.991650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8031af5-6858-4354-94eb-70ed18faaaf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:33.994360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c8031af5-6858-4354-94eb-70ed18faaaf7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:34.003457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:34.135627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8031af5-6858-4354-94eb-70ed18faaaf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:34.137339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8031af5-6858-4354-94eb-70ed18faaaf7 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:34.159917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 708/1409] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:43.607630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:44.157247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=42 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:49.313818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d929b-fc1c-4411-adf7-8593370e1500 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:49.314372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d929b-fc1c-4411-adf7-8593370e1500 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:49.314372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d929b-fc1c-4411-adf7-8593370e1500 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:49.315810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 702/1410] 10.222.0.64 () {54 vars in 918 bytes} [Wed Jul 20 08:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:49.322698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28e3b6-9e47-498a-b5c3-3066c8bfdc05 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:49.323207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28e3b6-9e47-498a-b5c3-3066c8bfdc05 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:49.323303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28e3b6-9e47-498a-b5c3-3066c8bfdc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:49.404720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ae28e3b6-9e47-498a-b5c3-3066c8bfdc05 None None] MFA Rules not processed for user `928054a577d14522bd0f00417f4f8c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:49.406014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ae28e3b6-9e47-498a-b5c3-3066c8bfdc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:49.521647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 709/1411] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Jul 20 08:18:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:49.539048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3da94d-f24d-49a8-bbf2-d7f7cbe64229 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:49.539339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3da94d-f24d-49a8-bbf2-d7f7cbe64229 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:49.540237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3da94d-f24d-49a8-bbf2-d7f7cbe64229 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:49.541763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 703/1412] 10.222.0.64 () {54 vars in 918 bytes} [Wed Jul 20 08:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:49.547206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daf21f-ef04-4f3d-a0cc-1f3d991826f0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:49.548063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daf21f-ef04-4f3d-a0cc-1f3d991826f0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:49.548063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daf21f-ef04-4f3d-a0cc-1f3d991826f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:49.626679 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:49.629928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-54daf21f-ef04-4f3d-a0cc-1f3d991826f0 None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:49.631735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-54daf21f-ef04-4f3d-a0cc-1f3d991826f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:49.764744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 710/1413] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Jul 20 08:18:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:52.366203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1355e0-88c8-4254-b7c7-f8f99a4c67b9 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:52.366807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1355e0-88c8-4254-b7c7-f8f99a4c67b9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:52.366807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1355e0-88c8-4254-b7c7-f8f99a4c67b9 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:52.368440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:52.374216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f76ca6-f827-4c55-a65f-03c7dca0a30f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:52.374726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f76ca6-f827-4c55-a65f-03c7dca0a30f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:52.374840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f76ca6-f827-4c55-a65f-03c7dca0a30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:52.449377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-37f76ca6-f827-4c55-a65f-03c7dca0a30f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:52.450657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-37f76ca6-f827-4c55-a65f-03c7dca0a30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:52.515233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:52.564222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:52.611583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1355e0-88c8-4254-b7c7-f8f99a4c67b9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:52.613317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-df1355e0-88c8-4254-b7c7-f8f99a4c67b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:52.772586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1355e0-88c8-4254-b7c7-f8f99a4c67b9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:52.852705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:52.895887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed91fff-eb94-44ee-95ec-bcf742158864 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:52.896536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed91fff-eb94-44ee-95ec-bcf742158864 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:52.896842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed91fff-eb94-44ee-95ec-bcf742158864 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:52.897115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed91fff-eb94-44ee-95ec-bcf742158864 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:52.898935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3ed91fff-eb94-44ee-95ec-bcf742158864 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:53.004991 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed91fff-eb94-44ee-95ec-bcf742158864 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:53.006715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed91fff-eb94-44ee-95ec-bcf742158864 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:53.029167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 705/1416] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:53.968697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830485b-e866-4816-9e32-c6c0c3efacce None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:53.969271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830485b-e866-4816-9e32-c6c0c3efacce None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:53.969271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830485b-e866-4816-9e32-c6c0c3efacce None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:53.970727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:53.976172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b24f2-4aa4-44c1-9cc9-d04d9288ddb2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:53.977442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b24f2-4aa4-44c1-9cc9-d04d9288ddb2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:53.977442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91b24f2-4aa4-44c1-9cc9-d04d9288ddb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:54.062243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b91b24f2-4aa4-44c1-9cc9-d04d9288ddb2 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:54.063530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b91b24f2-4aa4-44c1-9cc9-d04d9288ddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:54.092330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:54.171616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:54.324754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830485b-e866-4816-9e32-c6c0c3efacce None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:54.326309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2830485b-e866-4816-9e32-c6c0c3efacce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:54.484484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830485b-e866-4816-9e32-c6c0c3efacce None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:54.530421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:54.612236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d5f4d-30b7-4efe-af84-33a448a8da6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:54.612801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d5f4d-30b7-4efe-af84-33a448a8da6f service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:54.613086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d5f4d-30b7-4efe-af84-33a448a8da6f service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:54.613363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d5f4d-30b7-4efe-af84-33a448a8da6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:54.615125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0f9d5f4d-30b7-4efe-af84-33a448a8da6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:54.716584 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d5f4d-30b7-4efe-af84-33a448a8da6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:54.718609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d5f4d-30b7-4efe-af84-33a448a8da6f service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:54.739662 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 713/1419] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:55.656466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47675bb5-f223-4e34-b616-581030326489 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:55.657066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47675bb5-f223-4e34-b616-581030326489 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:55.657066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47675bb5-f223-4e34-b616-581030326489 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:55.658447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:55.665566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e53617-f07b-4e24-ab12-16755f255600 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:55.666126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e53617-f07b-4e24-ab12-16755f255600 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:55.666126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e53617-f07b-4e24-ab12-16755f255600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:55.741977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:55.771616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-15e53617-f07b-4e24-ab12-16755f255600 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:55.772883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-15e53617-f07b-4e24-ab12-16755f255600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:55.884562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:56.036602 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47675bb5-f223-4e34-b616-581030326489 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:56.038174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-47675bb5-f223-4e34-b616-581030326489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:56.189637 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47675bb5-f223-4e34-b616-581030326489 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:56.202482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:56.313784 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5c902-1e9e-46f4-94bb-4183256de47d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:56.314447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5c902-1e9e-46f4-94bb-4183256de47d service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:56.314767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5c902-1e9e-46f4-94bb-4183256de47d service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:56.315059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5c902-1e9e-46f4-94bb-4183256de47d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:56.316764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dbf5c902-1e9e-46f4-94bb-4183256de47d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:56.412955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5c902-1e9e-46f4-94bb-4183256de47d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:56.415269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5c902-1e9e-46f4-94bb-4183256de47d service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:56.437522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 708/1422] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:57.322668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106b80-5148-4009-8249-3e3e573d274e None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:57.323253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106b80-5148-4009-8249-3e3e573d274e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:57.323253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106b80-5148-4009-8249-3e3e573d274e None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:57.324615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:18:57.329912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07514373-739d-4990-8a25-98e406dbc7d6 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:57.330405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07514373-739d-4990-8a25-98e406dbc7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:57.330504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07514373-739d-4990-8a25-98e406dbc7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:57.349989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:57.422528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-07514373-739d-4990-8a25-98e406dbc7d6 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:57.424690 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-07514373-739d-4990-8a25-98e406dbc7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:57.525961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:57.683483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52106b80-5148-4009-8249-3e3e573d274e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:57.685223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-52106b80-5148-4009-8249-3e3e573d274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:57.812799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:57.855527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52106b80-5148-4009-8249-3e3e573d274e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:57.986215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f37dd8-8dbe-447a-9f8a-cb1ee6d1dd7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:57.986828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f37dd8-8dbe-447a-9f8a-cb1ee6d1dd7e service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:57.987115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f37dd8-8dbe-447a-9f8a-cb1ee6d1dd7e service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:57.987401 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f37dd8-8dbe-447a-9f8a-cb1ee6d1dd7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:57.989116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-73f37dd8-8dbe-447a-9f8a-cb1ee6d1dd7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:58.075308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:58.091122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f37dd8-8dbe-447a-9f8a-cb1ee6d1dd7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:58.092775 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f37dd8-8dbe-447a-9f8a-cb1ee6d1dd7e service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:58.111939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 716/1425] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:18:59.002237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9f644-24c4-48cb-970d-133776ec3b5e None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:59.002886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9f644-24c4-48cb-970d-133776ec3b5e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:59.002886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9f644-24c4-48cb-970d-133776ec3b5e None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:59.004223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:18:59.010975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd0d27-ffc8-4a2d-b230-a9ca15b3db52 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:59.011500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd0d27-ffc8-4a2d-b230-a9ca15b3db52 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:59.011603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd0d27-ffc8-4a2d-b230-a9ca15b3db52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:59.093321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f8dd0d27-ffc8-4a2d-b230-a9ca15b3db52 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:18:59.094553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f8dd0d27-ffc8-4a2d-b230-a9ca15b3db52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:59.196572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:18:59.357601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9f644-24c4-48cb-970d-133776ec3b5e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:18:59.359134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4eb9f644-24c4-48cb-970d-133776ec3b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:59.447414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:59.517380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9f644-24c4-48cb-970d-133776ec3b5e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:18:59.642990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a50f2-f77e-4498-a0fa-e245cdb5f3bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:18:59.643555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a50f2-f77e-4498-a0fa-e245cdb5f3bd service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:18:59.643832 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a50f2-f77e-4498-a0fa-e245cdb5f3bd service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:18:59.644114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a50f2-f77e-4498-a0fa-e245cdb5f3bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:18:59.645765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-955a50f2-f77e-4498-a0fa-e245cdb5f3bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:18:59.700477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:18:59.745980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a50f2-f77e-4498-a0fa-e245cdb5f3bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:18:59.747672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a50f2-f77e-4498-a0fa-e245cdb5f3bd service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:18:59.767007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 711/1428] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:00.599040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ba812-73c4-48d2-8922-1400f9d62776 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:00.599907 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ba812-73c4-48d2-8922-1400f9d62776 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:00.599907 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ba812-73c4-48d2-8922-1400f9d62776 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:00.601682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:00.608875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c593-7e5f-4d65-8e83-0afa915523f9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:00.609405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c593-7e5f-4d65-8e83-0afa915523f9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:00.609405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2c593-7e5f-4d65-8e83-0afa915523f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:00.697649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-16f2c593-7e5f-4d65-8e83-0afa915523f9 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:00.698929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-16f2c593-7e5f-4d65-8e83-0afa915523f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:00.820823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:00.987594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ba812-73c4-48d2-8922-1400f9d62776 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:00.989668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b55ba812-73c4-48d2-8922-1400f9d62776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:01.056537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:01.165367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ba812-73c4-48d2-8922-1400f9d62776 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:01.300598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84b8a1-8ae0-4a77-96a8-d26b67d324ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:01.301258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b8a1-8ae0-4a77-96a8-d26b67d324ba service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:01.301596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b8a1-8ae0-4a77-96a8-d26b67d324ba service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:01.301800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b8a1-8ae0-4a77-96a8-d26b67d324ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:01.303571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7b84b8a1-8ae0-4a77-96a8-d26b67d324ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:01.313408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:01.423477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84b8a1-8ae0-4a77-96a8-d26b67d324ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:01.425035 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84b8a1-8ae0-4a77-96a8-d26b67d324ba service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:01.456375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 719/1431] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:02.292520 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a8be-be6a-4e90-9013-1fda2833ff3f None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:02.293127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a8be-be6a-4e90-9013-1fda2833ff3f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:02.293127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c67a8be-be6a-4e90-9013-1fda2833ff3f None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:02.294345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:02.299518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f1bee-e579-44b2-ab93-ac8821fd9f34 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:02.300099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f1bee-e579-44b2-ab93-ac8821fd9f34 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:02.300099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f1bee-e579-44b2-ab93-ac8821fd9f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:02.381714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9d8f1bee-e579-44b2-ab93-ac8821fd9f34 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:02.383113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9d8f1bee-e579-44b2-ab93-ac8821fd9f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:02.456683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:02.500610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:02.653060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67a8be-be6a-4e90-9013-1fda2833ff3f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:02.654656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0c67a8be-be6a-4e90-9013-1fda2833ff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:02.672752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:02.823032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c67a8be-be6a-4e90-9013-1fda2833ff3f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:02.919283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:03.067471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a423ce3-82f3-4d75-9b45-1b57737e76ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:03.068122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a423ce3-82f3-4d75-9b45-1b57737e76ab service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:03.068405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a423ce3-82f3-4d75-9b45-1b57737e76ab service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:03.068678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a423ce3-82f3-4d75-9b45-1b57737e76ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:03.070959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8a423ce3-82f3-4d75-9b45-1b57737e76ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:03.165683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a423ce3-82f3-4d75-9b45-1b57737e76ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:03.167354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a423ce3-82f3-4d75-9b45-1b57737e76ab service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:03.186231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 714/1434] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:04.010478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb87031-afe5-4bfc-91fc-3de7d175a411 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:04.011103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb87031-afe5-4bfc-91fc-3de7d175a411 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:04.011103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb87031-afe5-4bfc-91fc-3de7d175a411 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:04.012423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:04.017627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa33a6-36d7-42e6-9b79-d3f0422593a0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:04.018121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa33a6-36d7-42e6-9b79-d3f0422593a0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:04.018346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa33a6-36d7-42e6-9b79-d3f0422593a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:04.099885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-59fa33a6-36d7-42e6-9b79-d3f0422593a0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:04.101179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-59fa33a6-36d7-42e6-9b79-d3f0422593a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:04.146920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:04.207170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:04.376084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb87031-afe5-4bfc-91fc-3de7d175a411 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:04.377615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9bb87031-afe5-4bfc-91fc-3de7d175a411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:04.540005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb87031-afe5-4bfc-91fc-3de7d175a411 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:04.598614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:04.660408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b8855-0b17-4cbe-b34e-493a6d9e4553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:04.660972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b8855-0b17-4cbe-b34e-493a6d9e4553 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:04.661259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b8855-0b17-4cbe-b34e-493a6d9e4553 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:04.661531 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b8855-0b17-4cbe-b34e-493a6d9e4553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:04.663238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-547b8855-0b17-4cbe-b34e-493a6d9e4553 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:04.757917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b8855-0b17-4cbe-b34e-493a6d9e4553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:04.759700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b8855-0b17-4cbe-b34e-493a6d9e4553 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:04.779906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 722/1437] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:05.608499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c3828-46f7-4e64-b2b3-7b85f292b1ca None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:05.609039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c3828-46f7-4e64-b2b3-7b85f292b1ca None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:05.609039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c3828-46f7-4e64-b2b3-7b85f292b1ca None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:05.610607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:05.615671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438358a6-e2e3-4d52-b89a-1d4ce3164180 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:05.616122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438358a6-e2e3-4d52-b89a-1d4ce3164180 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:05.616240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438358a6-e2e3-4d52-b89a-1d4ce3164180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:05.704580 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-438358a6-e2e3-4d52-b89a-1d4ce3164180 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:05.705833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-438358a6-e2e3-4d52-b89a-1d4ce3164180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:05.709078 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:05.811090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:05.965799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c3828-46f7-4e64-b2b3-7b85f292b1ca None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:05.967315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-811c3828-46f7-4e64-b2b3-7b85f292b1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:06.120045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c3828-46f7-4e64-b2b3-7b85f292b1ca None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:06.147892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:06.241536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f53d9-2921-4150-9e09-9a9e572c2852 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:06.242590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f53d9-2921-4150-9e09-9a9e572c2852 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:06.242926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f53d9-2921-4150-9e09-9a9e572c2852 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:06.243193 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f53d9-2921-4150-9e09-9a9e572c2852 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:06.244880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-159f53d9-2921-4150-9e09-9a9e572c2852 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:06.340502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f53d9-2921-4150-9e09-9a9e572c2852 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:06.342111 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f53d9-2921-4150-9e09-9a9e572c2852 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:06.361492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 717/1440] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:08.354294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d9123-03df-4722-95fa-2b33f22e041f None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:08.354924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d9123-03df-4722-95fa-2b33f22e041f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:08.354924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d9123-03df-4722-95fa-2b33f22e041f None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:08.356276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:08.361796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00601406-64bf-4737-892c-404179028184 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:08.362393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00601406-64bf-4737-892c-404179028184 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:08.362393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00601406-64bf-4737-892c-404179028184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:08.405638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:08.438445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-00601406-64bf-4737-892c-404179028184 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:08.439704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-00601406-64bf-4737-892c-404179028184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:08.550836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:08.591006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d9123-03df-4722-95fa-2b33f22e041f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:08.592600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ab7d9123-03df-4722-95fa-2b33f22e041f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:08.704754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d9123-03df-4722-95fa-2b33f22e041f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:08.737783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:08.782215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b79a8-7b67-49cb-b64c-127d559de008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:08.782899 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b79a8-7b67-49cb-b64c-127d559de008 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:08.783182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b79a8-7b67-49cb-b64c-127d559de008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:08.783457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b79a8-7b67-49cb-b64c-127d559de008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:08.785120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b60b79a8-7b67-49cb-b64c-127d559de008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:08.905096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b79a8-7b67-49cb-b64c-127d559de008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:08.906599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60b79a8-7b67-49cb-b64c-127d559de008 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:08.927474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 725/1443] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:11.147586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52b4f7-7eed-47f0-b9d1-ee7f3ff2fe4a None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:11.148260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52b4f7-7eed-47f0-b9d1-ee7f3ff2fe4a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:11.148489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52b4f7-7eed-47f0-b9d1-ee7f3ff2fe4a None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:11.149873 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:19:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:19:11.155291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fb967-b116-41ff-be70-f2cc48f0bc7f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:11.155738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fb967-b116-41ff-be70-f2cc48f0bc7f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:11.155865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fb967-b116-41ff-be70-f2cc48f0bc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:11.230180 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8d9fb967-b116-41ff-be70-f2cc48f0bc7f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:11.231455 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8d9fb967-b116-41ff-be70-f2cc48f0bc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:11.245764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:11.340123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:11.379322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52b4f7-7eed-47f0-b9d1-ee7f3ff2fe4a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:11.381073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5e52b4f7-7eed-47f0-b9d1-ee7f3ff2fe4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:11.494892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52b4f7-7eed-47f0-b9d1-ee7f3ff2fe4a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:11.507561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:11.578307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2467dbb-9ada-4c54-a074-7fc1f047d9b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:11.578996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2467dbb-9ada-4c54-a074-7fc1f047d9b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:11.579282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2467dbb-9ada-4c54-a074-7fc1f047d9b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:11.579564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2467dbb-9ada-4c54-a074-7fc1f047d9b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:11.581220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d2467dbb-9ada-4c54-a074-7fc1f047d9b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:11.693037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2467dbb-9ada-4c54-a074-7fc1f047d9b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:11.695084 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2467dbb-9ada-4c54-a074-7fc1f047d9b9 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:11.714888 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 720/1446] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:14.134793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b207baf-7319-4d76-a45a-c6f99ad6c43b None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:14.135311 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b207baf-7319-4d76-a45a-c6f99ad6c43b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:14.135403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b207baf-7319-4d76-a45a-c6f99ad6c43b None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:14.136751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:14.142494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d2092-bfb6-4e34-9a52-d1b0c6154ba5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:14.143107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d2092-bfb6-4e34-9a52-d1b0c6154ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:14.143107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69d2092-bfb6-4e34-9a52-d1b0c6154ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:14.192258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:14.214094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e69d2092-bfb6-4e34-9a52-d1b0c6154ba5 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:14.215360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e69d2092-bfb6-4e34-9a52-d1b0c6154ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:14.327361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:14.369097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b207baf-7319-4d76-a45a-c6f99ad6c43b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:14.370737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7b207baf-7319-4d76-a45a-c6f99ad6c43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:14.494176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b207baf-7319-4d76-a45a-c6f99ad6c43b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:14.536888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:14.575408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7572788-824a-4a26-afc6-b145eecdf284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:14.576060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7572788-824a-4a26-afc6-b145eecdf284 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:14.576349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7572788-824a-4a26-afc6-b145eecdf284 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:14.576631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7572788-824a-4a26-afc6-b145eecdf284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:14.578364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c7572788-824a-4a26-afc6-b145eecdf284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:14.695603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7572788-824a-4a26-afc6-b145eecdf284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:14.697079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7572788-824a-4a26-afc6-b145eecdf284 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:14.716659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 728/1449] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:16.950651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ff1f-551a-407e-846c-bc21d5ed81b9 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:16.951558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ff1f-551a-407e-846c-bc21d5ed81b9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:16.951558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ff1f-551a-407e-846c-bc21d5ed81b9 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:16.952614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:16.958404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64450-4245-4492-bcad-b406ece34e27 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:16.958970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64450-4245-4492-bcad-b406ece34e27 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:16.958970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad64450-4245-4492-bcad-b406ece34e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:17.031488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3ad64450-4245-4492-bcad-b406ece34e27 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:17.032742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3ad64450-4245-4492-bcad-b406ece34e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:17.055495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:17.137270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:19:17.177955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059ff1f-551a-407e-846c-bc21d5ed81b9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:17.179687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3059ff1f-551a-407e-846c-bc21d5ed81b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:17.290518 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059ff1f-551a-407e-846c-bc21d5ed81b9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:17.311044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:17.370298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216be856-c411-4863-96f4-d963538aca07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:17.371158 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216be856-c411-4863-96f4-d963538aca07 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:17.371543 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216be856-c411-4863-96f4-d963538aca07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:17.371801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216be856-c411-4863-96f4-d963538aca07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:17.373460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-216be856-c411-4863-96f4-d963538aca07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:17.490609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216be856-c411-4863-96f4-d963538aca07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:17.492498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216be856-c411-4863-96f4-d963538aca07 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:17.511428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 723/1452] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:19.773728 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cea913-e28e-4c79-8c9b-08d5a00a6774 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:19.774348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cea913-e28e-4c79-8c9b-08d5a00a6774 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:19.774457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cea913-e28e-4c79-8c9b-08d5a00a6774 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:19.776575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:19:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:19.784828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b163cb2-609e-4dda-b2e6-54f2b6701dfc None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:19.785433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b163cb2-609e-4dda-b2e6-54f2b6701dfc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:19.785433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b163cb2-609e-4dda-b2e6-54f2b6701dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:19.859215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:19.862607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7b163cb2-609e-4dda-b2e6-54f2b6701dfc None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:19.864481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7b163cb2-609e-4dda-b2e6-54f2b6701dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:19.976655 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:20.016547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cea913-e28e-4c79-8c9b-08d5a00a6774 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:20.018135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-55cea913-e28e-4c79-8c9b-08d5a00a6774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:20.139582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cea913-e28e-4c79-8c9b-08d5a00a6774 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:20.194998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:20.220772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24782ef0-e969-44b1-b779-455d21ab31df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:20.221434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24782ef0-e969-44b1-b779-455d21ab31df service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:20.221719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24782ef0-e969-44b1-b779-455d21ab31df service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:20.221994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24782ef0-e969-44b1-b779-455d21ab31df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:20.223754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-24782ef0-e969-44b1-b779-455d21ab31df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:20.345974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24782ef0-e969-44b1-b779-455d21ab31df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:20.347465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24782ef0-e969-44b1-b779-455d21ab31df service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:20.369041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 731/1455] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:22.567236 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eccd85-d7f8-4768-a070-59d6dd9e37fc None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:22.567782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eccd85-d7f8-4768-a070-59d6dd9e37fc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:22.567782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eccd85-d7f8-4768-a070-59d6dd9e37fc None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:22.569969 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:22.575882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897869dc-21bb-4d3c-8e21-993d35eeb405 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:22.576379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897869dc-21bb-4d3c-8e21-993d35eeb405 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:22.576498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897869dc-21bb-4d3c-8e21-993d35eeb405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:22.648717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-897869dc-21bb-4d3c-8e21-993d35eeb405 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:22.649945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-897869dc-21bb-4d3c-8e21-993d35eeb405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:22.681383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:22.757177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:22.796004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eccd85-d7f8-4768-a070-59d6dd9e37fc None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:22.797571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-09eccd85-d7f8-4768-a070-59d6dd9e37fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:22.913881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eccd85-d7f8-4768-a070-59d6dd9e37fc None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:22.945146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:22.993001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33381fb3-788e-4912-b5c5-8f6a5b9f83fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:22.993664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381fb3-788e-4912-b5c5-8f6a5b9f83fa service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:22.993952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381fb3-788e-4912-b5c5-8f6a5b9f83fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:22.994229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381fb3-788e-4912-b5c5-8f6a5b9f83fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:22.995949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-33381fb3-788e-4912-b5c5-8f6a5b9f83fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:23.113996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33381fb3-788e-4912-b5c5-8f6a5b9f83fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:23.115488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33381fb3-788e-4912-b5c5-8f6a5b9f83fa service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:23.135059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 726/1458] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:25.255057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a35a-a4e8-4047-b8bf-3bdef807ea76 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:25.255641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a35a-a4e8-4047-b8bf-3bdef807ea76 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:25.255641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629a35a-a4e8-4047-b8bf-3bdef807ea76 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:25.257719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:19:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:25.263558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11210a62-0f60-4063-aa54-c2463fa98a6e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:25.264171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11210a62-0f60-4063-aa54-c2463fa98a6e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:25.264171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11210a62-0f60-4063-aa54-c2463fa98a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:25.339676 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-11210a62-0f60-4063-aa54-c2463fa98a6e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:25.340950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-11210a62-0f60-4063-aa54-c2463fa98a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:25.351505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:25.459591 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:19:25.498737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a35a-a4e8-4047-b8bf-3bdef807ea76 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:25.500340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8629a35a-a4e8-4047-b8bf-3bdef807ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:25.615171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629a35a-a4e8-4047-b8bf-3bdef807ea76 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:25.674971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:25.696383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8c4849-f967-450a-94ab-79c8add39d49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:25.697074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c4849-f967-450a-94ab-79c8add39d49 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:25.697360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c4849-f967-450a-94ab-79c8add39d49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:25.697639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8c4849-f967-450a-94ab-79c8add39d49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:25.699459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-db8c4849-f967-450a-94ab-79c8add39d49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:25.818373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c4849-f967-450a-94ab-79c8add39d49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:25.820257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8c4849-f967-450a-94ab-79c8add39d49 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:25.839141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 734/1461] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:19:28.003130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d032d3-1c43-4828-9f4c-0c642b3e1db9 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:28.004032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d032d3-1c43-4828-9f4c-0c642b3e1db9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:28.004331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d032d3-1c43-4828-9f4c-0c642b3e1db9 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:28.005792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:19:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:28.012309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c26645-2aa9-41e8-8363-81dc17dc119d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:28.012844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c26645-2aa9-41e8-8363-81dc17dc119d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:28.012844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c26645-2aa9-41e8-8363-81dc17dc119d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:28.082369 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-27c26645-2aa9-41e8-8363-81dc17dc119d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:28.083638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-27c26645-2aa9-41e8-8363-81dc17dc119d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:28.128593 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:28.196138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:28.235678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d032d3-1c43-4828-9f4c-0c642b3e1db9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:28.237272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-25d032d3-1c43-4828-9f4c-0c642b3e1db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:28.360176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d032d3-1c43-4828-9f4c-0c642b3e1db9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:28.399439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:28.439454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6429dc-bc23-46c7-b304-8a50d1b79578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:28.440109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6429dc-bc23-46c7-b304-8a50d1b79578 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:28.440405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6429dc-bc23-46c7-b304-8a50d1b79578 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:28.440685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6429dc-bc23-46c7-b304-8a50d1b79578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:28.442366 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5e6429dc-bc23-46c7-b304-8a50d1b79578 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:28.570117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6429dc-bc23-46c7-b304-8a50d1b79578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:28.571855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6429dc-bc23-46c7-b304-8a50d1b79578 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:28.595762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 729/1464] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:30.717389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc979d4-28a1-4a68-b09f-91539fb193d9 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:30.718012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc979d4-28a1-4a68-b09f-91539fb193d9 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:30.718012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc979d4-28a1-4a68-b09f-91539fb193d9 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:30.719373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:30.724904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b085c-7de3-4a02-8b67-3510244bc14b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:30.726009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b085c-7de3-4a02-8b67-3510244bc14b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:30.726147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b085c-7de3-4a02-8b67-3510244bc14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:30.796564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b92b085c-7de3-4a02-8b67-3510244bc14b None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:30.797814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b92b085c-7de3-4a02-8b67-3510244bc14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:30.816946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:30.905589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:30.946136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc979d4-28a1-4a68-b09f-91539fb193d9 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:30.947795 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-efc979d4-28a1-4a68-b09f-91539fb193d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:31.069710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc979d4-28a1-4a68-b09f-91539fb193d9 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:31.141379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:31.149672 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03071c-e239-435d-bd9d-9c0f2f4fe00b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:31.150368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03071c-e239-435d-bd9d-9c0f2f4fe00b service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:31.150681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03071c-e239-435d-bd9d-9c0f2f4fe00b service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:31.150963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03071c-e239-435d-bd9d-9c0f2f4fe00b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:31.152655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ab03071c-e239-435d-bd9d-9c0f2f4fe00b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:31.269392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03071c-e239-435d-bd9d-9c0f2f4fe00b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:31.270890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03071c-e239-435d-bd9d-9c0f2f4fe00b service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:31.290170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 737/1467] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:33.427464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c625a2-77e4-445c-b44a-7c708a9b4976 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:33.428954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c625a2-77e4-445c-b44a-7c708a9b4976 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:33.428954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c625a2-77e4-445c-b44a-7c708a9b4976 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:33.430158 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:19:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:33.435992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd44b-e6fe-49f1-8e74-cd81ff31981e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:33.436477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd44b-e6fe-49f1-8e74-cd81ff31981e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:33.436580 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dd44b-e6fe-49f1-8e74-cd81ff31981e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:33.505997 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a18dd44b-e6fe-49f1-8e74-cd81ff31981e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:33.507351 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a18dd44b-e6fe-49f1-8e74-cd81ff31981e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:33.560635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:33.612313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:33.650983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c625a2-77e4-445c-b44a-7c708a9b4976 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:33.653745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-99c625a2-77e4-445c-b44a-7c708a9b4976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:33.769338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c625a2-77e4-445c-b44a-7c708a9b4976 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:33.822237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:33.865115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e7687-03c4-46f3-ad63-0fa0d46182b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:33.866533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e7687-03c4-46f3-ad63-0fa0d46182b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:33.866895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e7687-03c4-46f3-ad63-0fa0d46182b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:33.867439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e7687-03c4-46f3-ad63-0fa0d46182b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:33.869287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6a1e7687-03c4-46f3-ad63-0fa0d46182b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:34.002493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e7687-03c4-46f3-ad63-0fa0d46182b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:34.004011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e7687-03c4-46f3-ad63-0fa0d46182b0 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:34.025301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 732/1470] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:43.610426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=22 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:44.022575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=66 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:54.370304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc4c09-c869-4bf0-9bbf-78178b03b324 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:54.370925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc4c09-c869-4bf0-9bbf-78178b03b324 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:54.370925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc4c09-c869-4bf0-9bbf-78178b03b324 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:54.372313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 739/1471] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:54.378315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baebe27-0c13-420e-8a51-b22d2456a167 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:54.378885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baebe27-0c13-420e-8a51-b22d2456a167 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:54.379004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baebe27-0c13-420e-8a51-b22d2456a167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:54.454024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5baebe27-0c13-420e-8a51-b22d2456a167 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:54.455310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5baebe27-0c13-420e-8a51-b22d2456a167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:54.562241 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 733/1472] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:19:54.653805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d3882-87cc-4855-a419-19f6a344e392 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:54.654095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d3882-87cc-4855-a419-19f6a344e392 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:54.654375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7d3882-87cc-4855-a419-19f6a344e392 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:54.655831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 740/1473] 10.222.0.64 () {54 vars in 966 bytes} [Wed Jul 20 08:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:54.661097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019bdc6-7477-4883-b4f6-eb66b1855b47 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:54.661396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019bdc6-7477-4883-b4f6-eb66b1855b47 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:54.662349 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9019bdc6-7477-4883-b4f6-eb66b1855b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:54.747241 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9019bdc6-7477-4883-b4f6-eb66b1855b47 None None] MFA Rules not processed for user `3778997232d44cd3a3c4fb3d53e4db53`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:54.748529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9019bdc6-7477-4883-b4f6-eb66b1855b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:54.771900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:54.835212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 734/1474] 10.222.0.64 () {58 vars in 1063 bytes} [Wed Jul 20 08:19:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:54.842902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d3882-87cc-4855-a419-19f6a344e392 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:54.844449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-da7d3882-87cc-4855-a419-19f6a344e392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:54.964572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7d3882-87cc-4855-a419-19f6a344e392 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:55.047020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995042b-8d68-4bd5-a713-4ad85f3a75f8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3778997232d44cd3a3c4fb3d53e4db53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:55.047719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995042b-8d68-4bd5-a713-4ad85f3a75f8 service glance] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:55.048007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995042b-8d68-4bd5-a713-4ad85f3a75f8 service glance] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:55.048283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995042b-8d68-4bd5-a713-4ad85f3a75f8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:55.050112 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4995042b-8d68-4bd5-a713-4ad85f3a75f8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:55.124347 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:55.186288 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995042b-8d68-4bd5-a713-4ad85f3a75f8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:55.187805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4995042b-8d68-4bd5-a713-4ad85f3a75f8 service glance] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:55.209368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 741/1475] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:19:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:19:57.418823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52821178-a3a3-49d1-a8a6-fc3433b47d41 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:57.419499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52821178-a3a3-49d1-a8a6-fc3433b47d41 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:57.419499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52821178-a3a3-49d1-a8a6-fc3433b47d41 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:57.420760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 735/1476] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:19:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:19:57.426274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47b247-555f-4948-b91f-824f021fc406 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:57.426756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47b247-555f-4948-b91f-824f021fc406 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:57.426870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a47b247-555f-4948-b91f-824f021fc406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:57.496031 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2a47b247-555f-4948-b91f-824f021fc406 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:19:57.497270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2a47b247-555f-4948-b91f-824f021fc406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:57.601641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 742/1477] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:19:57.626068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52821178-a3a3-49d1-a8a6-fc3433b47d41 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:19:57.627650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-52821178-a3a3-49d1-a8a6-fc3433b47d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:19:57.763209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52821178-a3a3-49d1-a8a6-fc3433b47d41 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:19:57.820444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:19:57.877973 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06d6b31-3b3f-450d-813e-a3e754cf39c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:19:57.878689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d6b31-3b3f-450d-813e-a3e754cf39c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:19:57.878980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d6b31-3b3f-450d-813e-a3e754cf39c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:19:57.879259 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06d6b31-3b3f-450d-813e-a3e754cf39c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:19:57.880012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d6b31-3b3f-450d-813e-a3e754cf39c3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:19:57.882224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-f06d6b31-3b3f-450d-813e-a3e754cf39c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:19:57.882224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 20 08:19:57.882224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: Jul 20 08:19:57.883437 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06d6b31-3b3f-450d-813e-a3e754cf39c3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:19:57.904982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 736/1478] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Jul 20 08:19:57 2022] GET /identity/v3/projects => generated 2553 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:20:00.067109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bff2ff-c81e-4ec4-b187-b27fedb4cf06 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:00.067776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bff2ff-c81e-4ec4-b187-b27fedb4cf06 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:00.067776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bff2ff-c81e-4ec4-b187-b27fedb4cf06 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:00.069801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 743/1479] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:20:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:20:00.075352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7106ec-c54d-4290-9fc0-afdebb71a01f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:00.075861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7106ec-c54d-4290-9fc0-afdebb71a01f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:00.075971 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7106ec-c54d-4290-9fc0-afdebb71a01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:00.144026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8b7106ec-c54d-4290-9fc0-afdebb71a01f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:20:00.145251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8b7106ec-c54d-4290-9fc0-afdebb71a01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:00.249747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 737/1480] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:20:00.294261 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bff2ff-c81e-4ec4-b187-b27fedb4cf06 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:20:00.295856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-64bff2ff-c81e-4ec4-b187-b27fedb4cf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:00.306875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:20:00.451883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bff2ff-c81e-4ec4-b187-b27fedb4cf06 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:20:00.534900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:20:00.571388 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e7480-a407-43af-b705-6ac8a694e47b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:20:00.572033 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7480-a407-43af-b705-6ac8a694e47b service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:00.572316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7480-a407-43af-b705-6ac8a694e47b service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:00.572588 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7480-a407-43af-b705-6ac8a694e47b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:00.574256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8c3e7480-a407-43af-b705-6ac8a694e47b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:00.666427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e7480-a407-43af-b705-6ac8a694e47b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:20:00.668074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e7480-a407-43af-b705-6ac8a694e47b service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:20:00.687359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 744/1481] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:20:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:20:03.228210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5904ed6-436a-4a5c-a5c0-5dfd2da612de None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:03.228788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5904ed6-436a-4a5c-a5c0-5dfd2da612de None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:03.228788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5904ed6-436a-4a5c-a5c0-5dfd2da612de None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:03.230131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 738/1482] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:20:03.235712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a163462-8be6-4b28-8870-098a047e1b01 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:03.236165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a163462-8be6-4b28-8870-098a047e1b01 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:03.236365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a163462-8be6-4b28-8870-098a047e1b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:03.306395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9a163462-8be6-4b28-8870-098a047e1b01 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:20:03.307730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9a163462-8be6-4b28-8870-098a047e1b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:03.359586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:20:03.412480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 745/1483] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:20:03.450328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5904ed6-436a-4a5c-a5c0-5dfd2da612de None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:20:03.451899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c5904ed6-436a-4a5c-a5c0-5dfd2da612de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:03.477012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:20:03.561319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5904ed6-436a-4a5c-a5c0-5dfd2da612de None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:20:03.637176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a6115b-8a26-4b03-ab6a-a3f3eac639e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:20:03.637861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6115b-8a26-4b03-ab6a-a3f3eac639e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:03.638146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6115b-8a26-4b03-ab6a-a3f3eac639e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:03.638423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a6115b-8a26-4b03-ab6a-a3f3eac639e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:03.640133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-08a6115b-8a26-4b03-ab6a-a3f3eac639e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:03.718207 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:20:03.754649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6115b-8a26-4b03-ab6a-a3f3eac639e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:20:03.756981 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a6115b-8a26-4b03-ab6a-a3f3eac639e0 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:20:03.777406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 739/1484] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:20:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:20:05.956228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261697f-9632-4378-a60b-46ed44d57c4a None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:05.956805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261697f-9632-4378-a60b-46ed44d57c4a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:05.956805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c261697f-9632-4378-a60b-46ed44d57c4a None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:05.958978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 746/1485] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:20:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:20:05.964937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16d9d0-675a-4dac-9042-b638f60cbfcb None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:05.965472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16d9d0-675a-4dac-9042-b638f60cbfcb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:05.965571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16d9d0-675a-4dac-9042-b638f60cbfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:06.041142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ba16d9d0-675a-4dac-9042-b638f60cbfcb None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:20:06.042401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ba16d9d0-675a-4dac-9042-b638f60cbfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:06.142378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 740/1486] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:20:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:20:06.183336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261697f-9632-4378-a60b-46ed44d57c4a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:20:06.185093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c261697f-9632-4378-a60b-46ed44d57c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:06.298536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c261697f-9632-4378-a60b-46ed44d57c4a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:20:06.377344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:20:06.380456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67391e5-9965-4436-b53e-8f218b0be27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:20:06.381379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67391e5-9965-4436-b53e-8f218b0be27e service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:06.381745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67391e5-9965-4436-b53e-8f218b0be27e service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:06.382225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67391e5-9965-4436-b53e-8f218b0be27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:06.384195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e67391e5-9965-4436-b53e-8f218b0be27e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:06.505318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67391e5-9965-4436-b53e-8f218b0be27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:20:06.506993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67391e5-9965-4436-b53e-8f218b0be27e service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:20:06.527056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 747/1487] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:20:08.788013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b7bf4-d780-4631-85e6-a67fd1399c86 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:08.788609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b7bf4-d780-4631-85e6-a67fd1399c86 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:08.788609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b7bf4-d780-4631-85e6-a67fd1399c86 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:08.790022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 741/1488] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:20:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:20:08.795795 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbcda6-cdb9-4d69-a768-4ef4c389ccc7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:08.796234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbcda6-cdb9-4d69-a768-4ef4c389ccc7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:08.796384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbcda6-cdb9-4d69-a768-4ef4c389ccc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:08.865020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b2cbcda6-cdb9-4d69-a768-4ef4c389ccc7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:20:08.866320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b2cbcda6-cdb9-4d69-a768-4ef4c389ccc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:08.928014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:20:08.976024 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 748/1489] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:20:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:20:09.017979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b7bf4-d780-4631-85e6-a67fd1399c86 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:20:09.020883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8b5b7bf4-d780-4631-85e6-a67fd1399c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:09.061282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:20:09.132861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b7bf4-d780-4631-85e6-a67fd1399c86 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:20:09.210171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0b1c1-c968-4e67-ae1a-d4aafecadbf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:20:09.210854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0b1c1-c968-4e67-ae1a-d4aafecadbf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:09.211141 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0b1c1-c968-4e67-ae1a-d4aafecadbf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:09.211418 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0b1c1-c968-4e67-ae1a-d4aafecadbf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:09.213105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-42d0b1c1-c968-4e67-ae1a-d4aafecadbf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:09.308308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:20:09.340430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0b1c1-c968-4e67-ae1a-d4aafecadbf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:20:09.341799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0b1c1-c968-4e67-ae1a-d4aafecadbf4 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:20:09.361526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 742/1490] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:20:11.705420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d2bca-6c93-4869-9c52-3fe23b853f37 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:11.706007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d2bca-6c93-4869-9c52-3fe23b853f37 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:11.706138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d2bca-6c93-4869-9c52-3fe23b853f37 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:11.707769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 749/1491] 10.222.0.64 () {54 vars in 926 bytes} [Wed Jul 20 08:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:20:11.713944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e19d0-7469-412c-8828-939ba9d22c7a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:11.714501 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e19d0-7469-412c-8828-939ba9d22c7a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:11.714634 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e19d0-7469-412c-8828-939ba9d22c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:11.789533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-081e19d0-7469-412c-8828-939ba9d22c7a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:20:11.790909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-081e19d0-7469-412c-8828-939ba9d22c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:11.892773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 743/1492] 10.222.0.64 () {58 vars in 1023 bytes} [Wed Jul 20 08:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:20:11.934298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d2bca-6c93-4869-9c52-3fe23b853f37 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:20:11.935942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-613d2bca-6c93-4869-9c52-3fe23b853f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:12.049015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d2bca-6c93-4869-9c52-3fe23b853f37 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:20:12.134564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c67eac-0567-4599-bcd8-53c7574f3392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:20:12.135245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c67eac-0567-4599-bcd8-53c7574f3392 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:20:12.135536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c67eac-0567-4599-bcd8-53c7574f3392 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:20:12.135811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c67eac-0567-4599-bcd8-53c7574f3392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:20:12.137499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f0c67eac-0567-4599-bcd8-53c7574f3392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:20:12.147756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:20:12.256334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c67eac-0567-4599-bcd8-53c7574f3392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:20:12.257805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c67eac-0567-4599-bcd8-53c7574f3392 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:20:12.277738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 750/1493] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:20:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:20:21.890802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=82 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:20:22.275190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=50 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:41.201583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a3887-1f2a-40c8-86b9-1c6258557ed7 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:41.202177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a3887-1f2a-40c8-86b9-1c6258557ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:41.202177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a3887-1f2a-40c8-86b9-1c6258557ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:41.279243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d94a3887-1f2a-40c8-86b9-1c6258557ed7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:21:41.280387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d94a3887-1f2a-40c8-86b9-1c6258557ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:41.383349 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 744/1494] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:21:41.391751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc83c-928b-46ca-8ff1-0d72321afa24 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:41.392269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc83c-928b-46ca-8ff1-0d72321afa24 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:41.392441 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcc83c-928b-46ca-8ff1-0d72321afa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:41.469658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e9bcc83c-928b-46ca-8ff1-0d72321afa24 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:21:41.471442 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e9bcc83c-928b-46ca-8ff1-0d72321afa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:41.571155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 751/1495] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:21:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:21:41.576031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a3887-1f2a-40c8-86b9-1c6258557ed7 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:21:41.577243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d94a3887-1f2a-40c8-86b9-1c6258557ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:41.589373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a3887-1f2a-40c8-86b9-1c6258557ed7 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:21:41.654054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:41.692495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abea50f-029c-4e17-a692-993dcd6f3226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:21:41.693167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abea50f-029c-4e17-a692-993dcd6f3226 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:41.693450 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abea50f-029c-4e17-a692-993dcd6f3226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:41.693725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abea50f-029c-4e17-a692-993dcd6f3226 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:41.694460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abea50f-029c-4e17-a692-993dcd6f3226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:21:41.694798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abea50f-029c-4e17-a692-993dcd6f3226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:21:41.695838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-5abea50f-029c-4e17-a692-993dcd6f3226 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:21:41.695838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 20 08:21:41.695838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: Jul 20 08:21:41.696739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abea50f-029c-4e17-a692-993dcd6f3226 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:21:41.709755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 745/1496] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:21:41.718026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcc83c-928b-46ca-8ff1-0d72321afa24 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:21:41.728402 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcc83c-928b-46ca-8ff1-0d72321afa24 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:21:41.913040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:41.948671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a61379-905c-4110-a1c7-cb9e8118184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:21:41.949334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a61379-905c-4110-a1c7-cb9e8118184d admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:41.949616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a61379-905c-4110-a1c7-cb9e8118184d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:41.949889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a61379-905c-4110-a1c7-cb9e8118184d admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:41.950614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a61379-905c-4110-a1c7-cb9e8118184d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:21:41.952371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-87a61379-905c-4110-a1c7-cb9e8118184d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:21:41.952371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 20 08:21:41.952371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: Jul 20 08:21:41.953578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a61379-905c-4110-a1c7-cb9e8118184d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:21:42.009707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 752/1497] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:21:41 2022] POST /identity/v3/projects => generated 379 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:21:42.014023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abea50f-029c-4e17-a692-993dcd6f3226 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:21:42.016505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5abea50f-029c-4e17-a692-993dcd6f3226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:42.051858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abea50f-029c-4e17-a692-993dcd6f3226 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:21:42.151838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e97983-296b-4bf6-8444-bca5a9816345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:21:42.152605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e97983-296b-4bf6-8444-bca5a9816345 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:42.152887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e97983-296b-4bf6-8444-bca5a9816345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:42.153163 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e97983-296b-4bf6-8444-bca5a9816345 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:42.153857 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e97983-296b-4bf6-8444-bca5a9816345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:21:42.154918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-46e97983-296b-4bf6-8444-bca5a9816345 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:21:42.154918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 20 08:21:42.154918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: Jul 20 08:21:42.156124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e97983-296b-4bf6-8444-bca5a9816345 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:21:42.200622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 746/1498] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:21:42 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:21:42.204809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a61379-905c-4110-a1c7-cb9e8118184d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:21:42.207202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-87a61379-905c-4110-a1c7-cb9e8118184d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:42.271308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a61379-905c-4110-a1c7-cb9e8118184d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:21:42.359179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:42.371980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b63867-5c11-4c24-b3d5-939bcd5a90e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:21:42.372647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b63867-5c11-4c24-b3d5-939bcd5a90e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:42.372934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b63867-5c11-4c24-b3d5-939bcd5a90e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:42.373213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b63867-5c11-4c24-b3d5-939bcd5a90e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:42.373940 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b63867-5c11-4c24-b3d5-939bcd5a90e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:21:42.375003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-32b63867-5c11-4c24-b3d5-939bcd5a90e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:21:42.375003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 20 08:21:42.375003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: Jul 20 08:21:42.375895 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b63867-5c11-4c24-b3d5-939bcd5a90e5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:21:42.389758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 753/1499] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:21:42.393889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e97983-296b-4bf6-8444-bca5a9816345 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:21:42.395467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-46e97983-296b-4bf6-8444-bca5a9816345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:42.407392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e97983-296b-4bf6-8444-bca5a9816345 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:21:42.434398 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:42.509630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53514a-3af3-4591-b0f0-c02f097f3fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:21:42.510286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53514a-3af3-4591-b0f0-c02f097f3fcc admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:42.510612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53514a-3af3-4591-b0f0-c02f097f3fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:42.510922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53514a-3af3-4591-b0f0-c02f097f3fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:42.511694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53514a-3af3-4591-b0f0-c02f097f3fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:21:42.512721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-cb53514a-3af3-4591-b0f0-c02f097f3fcc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:21:42.512721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 20 08:21:42.512721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: Jul 20 08:21:42.513950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53514a-3af3-4591-b0f0-c02f097f3fcc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:21:42.526893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 747/1500] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:21:42.531357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b63867-5c11-4c24-b3d5-939bcd5a90e5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:21:42.532886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-32b63867-5c11-4c24-b3d5-939bcd5a90e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:42.568319 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b63867-5c11-4c24-b3d5-939bcd5a90e5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:21:42.663745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee0627-f07e-4d63-8b19-8c8519820a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:21:42.664631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee0627-f07e-4d63-8b19-8c8519820a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:42.664907 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee0627-f07e-4d63-8b19-8c8519820a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:42.665174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee0627-f07e-4d63-8b19-8c8519820a9b admin admin] PATH_INFO: `/v3/projects/cbaf3fa2318549e4bfb13edb07e43212/users/aa4dae1ac77e439b86391a0a3f93009b/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:42.692475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee0627-f07e-4d63-8b19-8c8519820a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbaf3fa2318549e4bfb13edb07e43212, user_id=aa4dae1ac77e439b86391a0a3f93009b, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:21:42.693529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-8cee0627-f07e-4d63-8b19-8c8519820a9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:21:42.693529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 20 08:21:42.693529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: Jul 20 08:21:42.695517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee0627-f07e-4d63-8b19-8c8519820a9b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:21:42.720308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 754/1501] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:21:42 2022] PUT /identity/v3/projects/cbaf3fa2318549e4bfb13edb07e43212/users/aa4dae1ac77e439b86391a0a3f93009b/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:21:42.733628 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf787a-f986-47c1-b964-cbb3c4fccc42 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:42.733921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf787a-f986-47c1-b964-cbb3c4fccc42 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:42.734209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf787a-f986-47c1-b964-cbb3c4fccc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:42.793024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-20bf787a-f986-47c1-b964-cbb3c4fccc42 None None] MFA Rules not processed for user `aa4dae1ac77e439b86391a0a3f93009b`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:21:42.794179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-20bf787a-f986-47c1-b964-cbb3c4fccc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:42.874929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 748/1502] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:21:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:21:42.888781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd384715-e5fd-43af-8700-85d324ba7c70 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:42.889116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd384715-e5fd-43af-8700-85d324ba7c70 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:42.889391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd384715-e5fd-43af-8700-85d324ba7c70 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:42.890829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 755/1503] 10.222.0.64 () {54 vars in 964 bytes} [Wed Jul 20 08:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:21:42.896331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce056-f7ec-43cb-abb0-bf7baf433632 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:42.896629 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce056-f7ec-43cb-abb0-bf7baf433632 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:42.896927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273ce056-f7ec-43cb-abb0-bf7baf433632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:42.929171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:42.976655 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-273ce056-f7ec-43cb-abb0-bf7baf433632 None None] MFA Rules not processed for user `41a647bd72f344fa963dcd3eb5434b6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:21:42.977891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-273ce056-f7ec-43cb-abb0-bf7baf433632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:43.059487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 749/1504] 10.222.0.64 () {58 vars in 1061 bytes} [Wed Jul 20 08:21:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:21:43.068188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd384715-e5fd-43af-8700-85d324ba7c70 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:21:43.069716 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bd384715-e5fd-43af-8700-85d324ba7c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:43.120469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:43.178268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd384715-e5fd-43af-8700-85d324ba7c70 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:21:43.252783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513236f6-86ac-4a75-a885-65f52d085a4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:21:43.253447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513236f6-86ac-4a75-a885-65f52d085a4e service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:43.253731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513236f6-86ac-4a75-a885-65f52d085a4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:43.254060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513236f6-86ac-4a75-a885-65f52d085a4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:43.255746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-513236f6-86ac-4a75-a885-65f52d085a4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:43.357694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513236f6-86ac-4a75-a885-65f52d085a4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:21:43.359155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513236f6-86ac-4a75-a885-65f52d085a4e service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:21:43.363588 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:43.380774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 756/1505] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:21:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:21:43.423583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6559a-489f-4702-b38f-edfd5ebbec55 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:43.424667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6559a-489f-4702-b38f-edfd5ebbec55 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:43.424935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6559a-489f-4702-b38f-edfd5ebbec55 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:43.426231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 750/1506] 10.222.0.64 () {52 vars in 829 bytes} [Wed Jul 20 08:21:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:21:43.429976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513236f6-86ac-4a75-a885-65f52d085a4e service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:21:43.431238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-513236f6-86ac-4a75-a885-65f52d085a4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:43.530453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513236f6-86ac-4a75-a885-65f52d085a4e service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:21:43.628404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:21:43.629192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:43.629474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:43.629749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] PATH_INFO: `/v3/users/aa4dae1ac77e439b86391a0a3f93009b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:43.643540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa4dae1ac77e439b86391a0a3f93009b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:21:43.644583 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:21:43.644583 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 20 08:21:43.644583 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: Jul 20 08:21:43.645814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:21:43.655847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:43.715363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa4dae1ac77e439b86391a0a3f93009b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:21:43.734734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa4dae1ac77e439b86391a0a3f93009b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:21:43.735312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa4dae1ac77e439b86391a0a3f93009b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:21:43.745995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa4dae1ac77e439b86391a0a3f93009b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:21:43.759198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa4dae1ac77e439b86391a0a3f93009b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:21:43.764848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 757/1507] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:21:43 2022] DELETE /identity/v3/users/aa4dae1ac77e439b86391a0a3f93009b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:21:43.769958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ad069-5ad9-41c6-bdd3-23f3d0ba93fe None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:43.770256 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ad069-5ad9-41c6-bdd3-23f3d0ba93fe None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:43.770564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ad069-5ad9-41c6-bdd3-23f3d0ba93fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:43.828576 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9b2ad069-5ad9-41c6-bdd3-23f3d0ba93fe None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:21:43.829803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9b2ad069-5ad9-41c6-bdd3-23f3d0ba93fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:43.911683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:43.926078 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 751/1508] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:21:43.935840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:21:43.937105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:44.042819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438f196a-1584-472b-b8b8-f95c73b82ae8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:21:44.099928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:44.121709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2c614-e87b-4c76-b9ee-6ab756fc2418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:21:44.122359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2c614-e87b-4c76-b9ee-6ab756fc2418 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:44.122674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2c614-e87b-4c76-b9ee-6ab756fc2418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:44.122956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2c614-e87b-4c76-b9ee-6ab756fc2418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:44.124617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-31b2c614-e87b-4c76-b9ee-6ab756fc2418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:44.237241 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2c614-e87b-4c76-b9ee-6ab756fc2418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:21:44.238652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2c614-e87b-4c76-b9ee-6ab756fc2418 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:21:44.260710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 758/1509] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:21:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:21:44.629063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ad069-5ad9-41c6-bdd3-23f3d0ba93fe None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:21:44.630457 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9b2ad069-5ad9-41c6-bdd3-23f3d0ba93fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:21:44.642258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ad069-5ad9-41c6-bdd3-23f3d0ba93fe None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:21:44.743526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:21:44.744395 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:21:44.744674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:21:44.744949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] PATH_INFO: `/v3/projects/cbaf3fa2318549e4bfb13edb07e43212` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:21:44.756093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbaf3fa2318549e4bfb13edb07e43212)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:21:44.757075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:21:44.757075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 20 08:21:44.757075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: Jul 20 08:21:44.758274 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:21:44.819380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbaf3fa2318549e4bfb13edb07e43212', 'request_id': 'req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:21:44.825599 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbaf3fa2318549e4bfb13edb07e43212', 'request_id': 'req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:21:44.838845 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbaf3fa2318549e4bfb13edb07e43212', 'request_id': 'req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:21:44.865399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] The token cache is being invalidate because project cbaf3fa2318549e4bfb13edb07e43212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:21:44.865690 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:21:44.866220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80c8b5bc-6b48-4ead-81b1-07dfacbfadcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:21:44.867873 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 752/1510] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:21:44 2022] DELETE /identity/v3/projects/cbaf3fa2318549e4bfb13edb07e43212 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:21:54.258290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=62 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:21:54.863284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:23:30.158233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa6404-ebe4-4600-9429-8099c94655db None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:30.158868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa6404-ebe4-4600-9429-8099c94655db None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:30.158974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa6404-ebe4-4600-9429-8099c94655db None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:30.160390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:23:30.166196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db25df-6954-4ba8-b92e-7876e76c81e2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:30.166722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db25df-6954-4ba8-b92e-7876e76c81e2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:30.166819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db25df-6954-4ba8-b92e-7876e76c81e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:30.255524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-55db25df-6954-4ba8-b92e-7876e76c81e2 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:23:30.256751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-55db25df-6954-4ba8-b92e-7876e76c81e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:30.366953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:23:30.522558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa6404-ebe4-4600-9429-8099c94655db None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:23:30.525613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5afa6404-ebe4-4600-9429-8099c94655db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:30.650987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa6404-ebe4-4600-9429-8099c94655db None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:23:30.656516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb0c1b-8eac-4f99-961c-df58ac994afd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:23:30.657172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb0c1b-8eac-4f99-961c-df58ac994afd service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:30.657453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb0c1b-8eac-4f99-961c-df58ac994afd service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:30.657728 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb0c1b-8eac-4f99-961c-df58ac994afd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:30.667395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb0c1b-8eac-4f99-961c-df58ac994afd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:23:30.669059 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb0c1b-8eac-4f99-961c-df58ac994afd service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:23:30.679487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 760/1513] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:23:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:23:31.541617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431bd3-2b42-47a9-9193-67754eeecbf7 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:31.542218 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431bd3-2b42-47a9-9193-67754eeecbf7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:31.542218 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76431bd3-2b42-47a9-9193-67754eeecbf7 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:31.543858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:23:31.549338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949561-c149-46bb-a318-98a8264aac17 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:31.549946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949561-c149-46bb-a318-98a8264aac17 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:31.549946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00949561-c149-46bb-a318-98a8264aac17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:31.570476 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-00949561-c149-46bb-a318-98a8264aac17 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:23:31.570760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed58fca-5b55-48f6-b0e2-c21f00a997b9 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:31.571469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed58fca-5b55-48f6-b0e2-c21f00a997b9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:31.571763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed58fca-5b55-48f6-b0e2-c21f00a997b9 None None] PATH_INFO: `/v3` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:31.572953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-00949561-c149-46bb-a318-98a8264aac17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:31.573467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 755/1515] 10.222.0.18 () {54 vars in 925 bytes} [Wed Jul 20 08:23:31 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:23:31.579842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49393b0-6eb0-441e-933a-b2357d72ff2a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:31.580132 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49393b0-6eb0-441e-933a-b2357d72ff2a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:31.580413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49393b0-6eb0-441e-933a-b2357d72ff2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:31.582334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 761/1516] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:23:31.650646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d49393b0-6eb0-441e-933a-b2357d72ff2a None None] MFA Rules not processed for user `928054a577d14522bd0f00417f4f8c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:23:31.651876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d49393b0-6eb0-441e-933a-b2357d72ff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:31.686282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:23:31.732286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 756/1517] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Jul 20 08:23:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:23:31.739287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00949561-c149-46bb-a318-98a8264aac17 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:23:31.752256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00949561-c149-46bb-a318-98a8264aac17 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:23:31.757753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f635705-4653-4ed3-8353-a1c776934565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:23:31.758990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f635705-4653-4ed3-8353-a1c776934565 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:31.759240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f635705-4653-4ed3-8353-a1c776934565 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:31.759528 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f635705-4653-4ed3-8353-a1c776934565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:31.767926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f635705-4653-4ed3-8353-a1c776934565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:23:31.769531 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f635705-4653-4ed3-8353-a1c776934565 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:23:31.780021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 762/1518] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:23:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:23:40.607961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0642f-0318-4ea4-85c1-0c905af2c4f4 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:40.608472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0642f-0318-4ea4-85c1-0c905af2c4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:40.608472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0642f-0318-4ea4-85c1-0c905af2c4f4 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:40.609804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 757/1519] 10.222.0.64 () {54 vars in 961 bytes} [Wed Jul 20 08:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:23:40.614814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539542f-7823-4d8a-88e7-7a07f044861c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:40.615224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539542f-7823-4d8a-88e7-7a07f044861c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:40.615375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539542f-7823-4d8a-88e7-7a07f044861c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:40.633543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a539542f-7823-4d8a-88e7-7a07f044861c None None] MFA Rules not processed for user `928054a577d14522bd0f00417f4f8c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:23:40.635337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a539542f-7823-4d8a-88e7-7a07f044861c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:40.644556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 763/1520] 10.222.0.64 () {58 vars in 1058 bytes} [Wed Jul 20 08:23:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:23:40.652856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0642f-0318-4ea4-85c1-0c905af2c4f4 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:23:40.666492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0642f-0318-4ea4-85c1-0c905af2c4f4 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:23:40.674615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15047e32-400c-4cf9-8971-2b7f53d628fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:23:40.674615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15047e32-400c-4cf9-8971-2b7f53d628fe service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:40.674615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15047e32-400c-4cf9-8971-2b7f53d628fe service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:40.674615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15047e32-400c-4cf9-8971-2b7f53d628fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:40.682013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15047e32-400c-4cf9-8971-2b7f53d628fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:23:40.683706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15047e32-400c-4cf9-8971-2b7f53d628fe service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:23:40.695521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 758/1521] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:23:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:23:43.001669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74fd0b-fb2a-4d01-a9b7-fa428dc970e5 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:43.002204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74fd0b-fb2a-4d01-a9b7-fa428dc970e5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:43.002204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74fd0b-fb2a-4d01-a9b7-fa428dc970e5 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:43.003606 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:23:43.008791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43151f44-bdc3-49f0-ad67-0296e8c673ad None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:43.009281 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43151f44-bdc3-49f0-ad67-0296e8c673ad None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:43.009445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43151f44-bdc3-49f0-ad67-0296e8c673ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:43.028865 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-43151f44-bdc3-49f0-ad67-0296e8c673ad None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:23:43.030084 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-43151f44-bdc3-49f0-ad67-0296e8c673ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:43.038323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:23:43.193613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74fd0b-fb2a-4d01-a9b7-fa428dc970e5 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:23:43.208626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74fd0b-fb2a-4d01-a9b7-fa428dc970e5 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:23:43.214077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502bd14-dba4-44f1-9e9e-764520f80519 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:23:43.214747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502bd14-dba4-44f1-9e9e-764520f80519 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:43.215035 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502bd14-dba4-44f1-9e9e-764520f80519 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:43.215313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502bd14-dba4-44f1-9e9e-764520f80519 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:43.224358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502bd14-dba4-44f1-9e9e-764520f80519 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:23:43.226069 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502bd14-dba4-44f1-9e9e-764520f80519 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:23:43.236858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 765/1524] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:23:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:23:44.098075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef85cf-9897-4107-aa57-d9d6f3d95b28 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:44.098650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef85cf-9897-4107-aa57-d9d6f3d95b28 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:44.098650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef85cf-9897-4107-aa57-d9d6f3d95b28 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:44.100015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 08:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:23:44.104910 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3be7a9-84a2-445c-b595-dd9dc9db038a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:44.105338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3be7a9-84a2-445c-b595-dd9dc9db038a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:44.105457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3be7a9-84a2-445c-b595-dd9dc9db038a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:44.124113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-cb3be7a9-84a2-445c-b595-dd9dc9db038a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:23:44.125326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cb3be7a9-84a2-445c-b595-dd9dc9db038a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:44.133210 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 08:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:23:44.286291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef85cf-9897-4107-aa57-d9d6f3d95b28 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:23:44.300617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef85cf-9897-4107-aa57-d9d6f3d95b28 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:23:44.306120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f72b7d0-ceaa-4d7d-b72f-69347199bfd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:23:44.306796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72b7d0-ceaa-4d7d-b72f-69347199bfd7 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:44.307084 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72b7d0-ceaa-4d7d-b72f-69347199bfd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:44.307643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f72b7d0-ceaa-4d7d-b72f-69347199bfd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:44.316146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72b7d0-ceaa-4d7d-b72f-69347199bfd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:23:44.318039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f72b7d0-ceaa-4d7d-b72f-69347199bfd7 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:23:44.328518 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 761/1527] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:23:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:23:48.528356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8370d4-ce89-40cf-aff2-13dcc8af692f None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:48.528991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8370d4-ce89-40cf-aff2-13dcc8af692f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:48.528991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8370d4-ce89-40cf-aff2-13dcc8af692f None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:48.530337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:23:48.535600 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b450e2-32fa-4767-ba1c-fbe3fc12fa47 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:48.536076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b450e2-32fa-4767-ba1c-fbe3fc12fa47 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:48.536181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b450e2-32fa-4767-ba1c-fbe3fc12fa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:48.555413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-04b450e2-32fa-4767-ba1c-fbe3fc12fa47 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:23:48.556638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-04b450e2-32fa-4767-ba1c-fbe3fc12fa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:48.564716 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:23:48.640405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8370d4-ce89-40cf-aff2-13dcc8af692f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:23:48.642152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dd8370d4-ce89-40cf-aff2-13dcc8af692f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:48.709857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:23:48.725964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8370d4-ce89-40cf-aff2-13dcc8af692f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:23:48.729488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36063a-abf4-4062-b145-6e995910bd7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:23:48.730229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36063a-abf4-4062-b145-6e995910bd7b service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:48.730514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36063a-abf4-4062-b145-6e995910bd7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:48.730825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36063a-abf4-4062-b145-6e995910bd7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:48.740018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36063a-abf4-4062-b145-6e995910bd7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:23:48.741466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36063a-abf4-4062-b145-6e995910bd7b service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:23:48.751792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 768/1530] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:23:50.864074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53ffd0-05b0-407e-a0c6-4a600c0fb59f None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:50.864632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53ffd0-05b0-407e-a0c6-4a600c0fb59f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:50.864632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53ffd0-05b0-407e-a0c6-4a600c0fb59f None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:50.866007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:23:50.871741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec854975-3baf-4b8b-a78d-5aeaadbfb119 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:50.872271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec854975-3baf-4b8b-a78d-5aeaadbfb119 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:50.872380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec854975-3baf-4b8b-a78d-5aeaadbfb119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:50.890373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ec854975-3baf-4b8b-a78d-5aeaadbfb119 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:23:50.891610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ec854975-3baf-4b8b-a78d-5aeaadbfb119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:50.899584 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:23:50.937431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53ffd0-05b0-407e-a0c6-4a600c0fb59f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:23:50.948696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53ffd0-05b0-407e-a0c6-4a600c0fb59f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:23:50.952347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ea944-4ba8-4478-91d5-26e045ac2d65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:23:50.953022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ea944-4ba8-4478-91d5-26e045ac2d65 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:50.953308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ea944-4ba8-4478-91d5-26e045ac2d65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:50.953589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ea944-4ba8-4478-91d5-26e045ac2d65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:50.962970 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ea944-4ba8-4478-91d5-26e045ac2d65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:23:50.964926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ea944-4ba8-4478-91d5-26e045ac2d65 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:23:50.974789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 764/1533] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:23:54.667612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a450d-68d9-4ce6-a2fe-63f9ccc7ab24 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:54.668203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a450d-68d9-4ce6-a2fe-63f9ccc7ab24 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:54.668203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a450d-68d9-4ce6-a2fe-63f9ccc7ab24 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:54.669531 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:23:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:23:54.677872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959251c-0596-46e8-a546-7eb92537e9eb None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:54.678408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959251c-0596-46e8-a546-7eb92537e9eb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:54.678521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959251c-0596-46e8-a546-7eb92537e9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:54.697932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d959251c-0596-46e8-a546-7eb92537e9eb None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:23:54.699715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d959251c-0596-46e8-a546-7eb92537e9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:54.707883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:23:55.028438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a450d-68d9-4ce6-a2fe-63f9ccc7ab24 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:23:55.030160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-af7a450d-68d9-4ce6-a2fe-63f9ccc7ab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:55.118189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a450d-68d9-4ce6-a2fe-63f9ccc7ab24 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:23:55.122698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0686985-d7a4-41ad-9475-1fe7bb87e3b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:23:55.123484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0686985-d7a4-41ad-9475-1fe7bb87e3b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:55.123871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0686985-d7a4-41ad-9475-1fe7bb87e3b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:55.124258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0686985-d7a4-41ad-9475-1fe7bb87e3b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:55.134148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0686985-d7a4-41ad-9475-1fe7bb87e3b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:23:55.136836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0686985-d7a4-41ad-9475-1fe7bb87e3b9 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:23:55.146993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 771/1536] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:23:57.284687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3e905-4a78-496c-afa5-6ab811cb4577 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:57.285196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3e905-4a78-496c-afa5-6ab811cb4577 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:57.285196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3e905-4a78-496c-afa5-6ab811cb4577 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:57.286621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 08:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:23:57.292704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3e4f7-0fc1-4dd9-9525-f95c239b3839 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:57.293171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3e4f7-0fc1-4dd9-9525-f95c239b3839 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:57.293292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3e4f7-0fc1-4dd9-9525-f95c239b3839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:57.312344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-66f3e4f7-0fc1-4dd9-9525-f95c239b3839 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:23:57.313567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-66f3e4f7-0fc1-4dd9-9525-f95c239b3839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:57.321560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 08:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:23:57.629729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3e905-4a78-496c-afa5-6ab811cb4577 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:23:57.631558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a7a3e905-4a78-496c-afa5-6ab811cb4577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:23:57.646080 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3e905-4a78-496c-afa5-6ab811cb4577 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:23:57.649622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac4a54-d905-4de8-8711-803e65d7dbbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:23:57.650266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac4a54-d905-4de8-8711-803e65d7dbbb service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:23:57.650556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac4a54-d905-4de8-8711-803e65d7dbbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:23:57.650906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac4a54-d905-4de8-8711-803e65d7dbbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:23:57.660912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac4a54-d905-4de8-8711-803e65d7dbbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:23:57.662246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ac4a54-d905-4de8-8711-803e65d7dbbb service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:23:57.672704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 767/1539] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:23:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:24:07.306662 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=50 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:24:07.668432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=50 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:31.875671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ceac88-40bf-4b95-b788-1320cb474dcf None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:31.879121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ceac88-40bf-4b95-b788-1320cb474dcf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:31.879121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ceac88-40bf-4b95-b788-1320cb474dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:31.909154 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-05ceac88-40bf-4b95-b788-1320cb474dcf None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:31.910618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-05ceac88-40bf-4b95-b788-1320cb474dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:31.919384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 773/1540] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:25:31.929942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b38eba-a71e-4114-a025-b446401da0b3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:31.930571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b38eba-a71e-4114-a025-b446401da0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:31.930571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b38eba-a71e-4114-a025-b446401da0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:31.968033 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-49b38eba-a71e-4114-a025-b446401da0b3 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:31.969976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-49b38eba-a71e-4114-a025-b446401da0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:31.985592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 768/1541] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:31.990819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ceac88-40bf-4b95-b788-1320cb474dcf None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:32.007888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ceac88-40bf-4b95-b788-1320cb474dcf None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:32.013426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f12792-04e2-40e7-84bc-654795a7ecec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:32.014459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f12792-04e2-40e7-84bc-654795a7ecec admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:32.014986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f12792-04e2-40e7-84bc-654795a7ecec admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:32.015549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f12792-04e2-40e7-84bc-654795a7ecec admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:32.017440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f12792-04e2-40e7-84bc-654795a7ecec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:25:32.018177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f12792-04e2-40e7-84bc-654795a7ecec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:32.023785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-c6f12792-04e2-40e7-84bc-654795a7ecec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:25:32.023785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 20 08:25:32.023785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: Jul 20 08:25:32.025419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f12792-04e2-40e7-84bc-654795a7ecec admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:32.046217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 774/1542] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:25:32.055850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b38eba-a71e-4114-a025-b446401da0b3 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:32.080760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b38eba-a71e-4114-a025-b446401da0b3 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:32.098713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2a24-f4be-40d6-98c9-552db81abad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:32.100881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2a24-f4be-40d6-98c9-552db81abad1 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:32.101876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2a24-f4be-40d6-98c9-552db81abad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:32.102714 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2a24-f4be-40d6-98c9-552db81abad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:32.104464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd2a24-f4be-40d6-98c9-552db81abad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:32.106976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-b9bd2a24-f4be-40d6-98c9-552db81abad1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:25:32.106976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 20 08:25:32.106976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: Jul 20 08:25:32.110460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd2a24-f4be-40d6-98c9-552db81abad1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:32.213680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 769/1543] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:32 2022] POST /identity/v3/projects => generated 391 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:32.217714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f12792-04e2-40e7-84bc-654795a7ecec admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:32.237067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f12792-04e2-40e7-84bc-654795a7ecec admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:32.329831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690682bc-44d0-41a7-9865-e12c0b389fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:32.330697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690682bc-44d0-41a7-9865-e12c0b389fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:32.330997 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690682bc-44d0-41a7-9865-e12c0b389fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:32.331274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690682bc-44d0-41a7-9865-e12c0b389fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:32.332011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690682bc-44d0-41a7-9865-e12c0b389fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:32.333220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-690682bc-44d0-41a7-9865-e12c0b389fd8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:25:32.333220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 20 08:25:32.333220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: Jul 20 08:25:32.335129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690682bc-44d0-41a7-9865-e12c0b389fd8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:32.371989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 775/1544] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:32 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:32.378898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2a24-f4be-40d6-98c9-552db81abad1 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:32.438100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2a24-f4be-40d6-98c9-552db81abad1 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:32.459728 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3453f-c85a-460a-ad52-b4b3a6435acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:32.461116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3453f-c85a-460a-ad52-b4b3a6435acd admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:32.461482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3453f-c85a-460a-ad52-b4b3a6435acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:32.462259 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3453f-c85a-460a-ad52-b4b3a6435acd admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:32.467108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3453f-c85a-460a-ad52-b4b3a6435acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:32.475284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3453f-c85a-460a-ad52-b4b3a6435acd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:32.506375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 770/1545] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:32.510124 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690682bc-44d0-41a7-9865-e12c0b389fd8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:32.534382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690682bc-44d0-41a7-9865-e12c0b389fd8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:32.547153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f393a-52aa-4d43-8556-9cc4299c4024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:32.551919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f393a-52aa-4d43-8556-9cc4299c4024 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:32.552749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f393a-52aa-4d43-8556-9cc4299c4024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:32.553301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f393a-52aa-4d43-8556-9cc4299c4024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:32.554348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ec47e-fc6a-47e0-b280-616267dec3d7 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:32.554792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f393a-52aa-4d43-8556-9cc4299c4024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:32.555075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ec47e-fc6a-47e0-b280-616267dec3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:32.555577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ec47e-fc6a-47e0-b280-616267dec3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:32.557298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f393a-52aa-4d43-8556-9cc4299c4024 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:32.582318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 776/1546] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:32.587479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f393a-52aa-4d43-8556-9cc4299c4024 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:32.603561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f393a-52aa-4d43-8556-9cc4299c4024 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:32.615743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf1bb47-65bf-4829-b9ad-634984c1d99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:32.617131 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1bb47-65bf-4829-b9ad-634984c1d99c admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:32.617394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0e9ec47e-fc6a-47e0-b280-616267dec3d7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:32.617903 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1bb47-65bf-4829-b9ad-634984c1d99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:32.618426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf1bb47-65bf-4829-b9ad-634984c1d99c admin admin] PATH_INFO: `/v3/projects/b044b1d7d5474351bd94eb1c90882933/users/bdc9e59d03334b72a8052e0f1112523f/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:32.619444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0e9ec47e-fc6a-47e0-b280-616267dec3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:32.648223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1bb47-65bf-4829-b9ad-634984c1d99c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b044b1d7d5474351bd94eb1c90882933, user_id=bdc9e59d03334b72a8052e0f1112523f, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:32.649575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 771/1547] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:32.652364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf1bb47-65bf-4829-b9ad-634984c1d99c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:32.678615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e06278-538f-4ece-a7d6-9d71d12cfcf8 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:32.679104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e06278-538f-4ece-a7d6-9d71d12cfcf8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:32.679499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e06278-538f-4ece-a7d6-9d71d12cfcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:32.689114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 777/1548] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:32 2022] PUT /identity/v3/projects/b044b1d7d5474351bd94eb1c90882933/users/bdc9e59d03334b72a8052e0f1112523f/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:25:32.691597 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874de298-4482-49d2-97c3-6ae11b87667b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:32.692222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874de298-4482-49d2-97c3-6ae11b87667b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:32.692737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874de298-4482-49d2-97c3-6ae11b87667b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:32.730011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-874de298-4482-49d2-97c3-6ae11b87667b None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:32.730011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-874de298-4482-49d2-97c3-6ae11b87667b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:32.760472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-09e06278-538f-4ece-a7d6-9d71d12cfcf8 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:32.763593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-09e06278-538f-4ece-a7d6-9d71d12cfcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:32.960378 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 778/1549] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:25:32.971252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271d11-93f4-4454-811f-a95c6918f58a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:32.971822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271d11-93f4-4454-811f-a95c6918f58a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:32.972289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75271d11-93f4-4454-811f-a95c6918f58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.044924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-75271d11-93f4-4454-811f-a95c6918f58a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:33.046737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-75271d11-93f4-4454-811f-a95c6918f58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:33.071329 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 779/1550] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:33.073060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd323d39-c898-4919-8da6-5d9b420a42af None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.073380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd323d39-c898-4919-8da6-5d9b420a42af None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.073694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd323d39-c898-4919-8da6-5d9b420a42af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.076298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 772/1551] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:33.077598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e06278-538f-4ece-a7d6-9d71d12cfcf8 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:33.084448 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-09e06278-538f-4ece-a7d6-9d71d12cfcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:33.120084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bd323d39-c898-4919-8da6-5d9b420a42af None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:33.121756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bd323d39-c898-4919-8da6-5d9b420a42af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:33.140837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 780/1552] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:33.142857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd323d39-c898-4919-8da6-5d9b420a42af None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:33.166465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd323d39-c898-4919-8da6-5d9b420a42af None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:33.181304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606cce3-8432-4552-8877-60cdfb831991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:33.181998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606cce3-8432-4552-8877-60cdfb831991 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.182294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606cce3-8432-4552-8877-60cdfb831991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.182544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606cce3-8432-4552-8877-60cdfb831991 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.183486 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606cce3-8432-4552-8877-60cdfb831991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:25:33.183800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606cce3-8432-4552-8877-60cdfb831991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:33.185488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606cce3-8432-4552-8877-60cdfb831991 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:33.214496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 781/1553] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:25:33.219447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2aa7-bb7c-4c83-956d-b33c288632be None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.219447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2aa7-bb7c-4c83-956d-b33c288632be None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.219447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba2aa7-bb7c-4c83-956d-b33c288632be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.280640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2cba2aa7-bb7c-4c83-956d-b33c288632be None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:33.281920 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2cba2aa7-bb7c-4c83-956d-b33c288632be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:33.321359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e06278-538f-4ece-a7d6-9d71d12cfcf8 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:33.328523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 782/1554] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:33.328523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ff014-c93d-420c-92a9-aecc8392cb8d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.328523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ff014-c93d-420c-92a9-aecc8392cb8d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.328523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ff014-c93d-420c-92a9-aecc8392cb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.335612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a18004-5ae3-4a8c-bf44-c8c51ef827c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:33.336310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a18004-5ae3-4a8c-bf44-c8c51ef827c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.336606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a18004-5ae3-4a8c-bf44-c8c51ef827c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.337147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a18004-5ae3-4a8c-bf44-c8c51ef827c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.345465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:33.362604 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a18004-5ae3-4a8c-bf44-c8c51ef827c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:33.366360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a18004-5ae3-4a8c-bf44-c8c51ef827c3 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:33.400178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-258ff014-c93d-420c-92a9-aecc8392cb8d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:33.402433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-258ff014-c93d-420c-92a9-aecc8392cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:33.406158 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 773/1555] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:33.409196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a18004-5ae3-4a8c-bf44-c8c51ef827c3 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:33.429291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 783/1556] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:33.429866 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a18004-5ae3-4a8c-bf44-c8c51ef827c3 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:33.430735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb1401-aa49-4ec1-ba29-2b144545cad0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.431084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb1401-aa49-4ec1-ba29-2b144545cad0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.431377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb1401-aa49-4ec1-ba29-2b144545cad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.443408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e8af-de60-4da8-8e90-bc50ae844397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:33.444107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e8af-de60-4da8-8e90-bc50ae844397 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.444497 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e8af-de60-4da8-8e90-bc50ae844397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.444763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265e8af-de60-4da8-8e90-bc50ae844397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.445499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265e8af-de60-4da8-8e90-bc50ae844397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:33.447791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265e8af-de60-4da8-8e90-bc50ae844397 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:33.483721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e1bb1401-aa49-4ec1-ba29-2b144545cad0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:33.486553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e1bb1401-aa49-4ec1-ba29-2b144545cad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:33.492294 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 774/1557] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:33.497878 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e8af-de60-4da8-8e90-bc50ae844397 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:33.585228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:33.660168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265e8af-de60-4da8-8e90-bc50ae844397 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:33.666339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ee507-e07a-4a25-bf38-910bb4a2309d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:33.667013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 784/1558] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:33.667582 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ee507-e07a-4a25-bf38-910bb4a2309d admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.668079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ee507-e07a-4a25-bf38-910bb4a2309d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.668536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ee507-e07a-4a25-bf38-910bb4a2309d admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.669533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ee507-e07a-4a25-bf38-910bb4a2309d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:25:33.670023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ee507-e07a-4a25-bf38-910bb4a2309d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:33.670545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbf39e-5221-48ff-a679-ef6bbf80582a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.670871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbf39e-5221-48ff-a679-ef6bbf80582a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.671207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbf39e-5221-48ff-a679-ef6bbf80582a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.673617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ee507-e07a-4a25-bf38-910bb4a2309d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:33.691827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 775/1559] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:25:33.701105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ee507-e07a-4a25-bf38-910bb4a2309d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:33.714876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ee507-e07a-4a25-bf38-910bb4a2309d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:33.721334 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8afb5b-cfb6-4e06-98ce-3ea11d44c114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:33.722479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8afb5b-cfb6-4e06-98ce-3ea11d44c114 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.722479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8afb5b-cfb6-4e06-98ce-3ea11d44c114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.722755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8afb5b-cfb6-4e06-98ce-3ea11d44c114 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.723353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8afb5b-cfb6-4e06-98ce-3ea11d44c114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:25:33.723664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8afb5b-cfb6-4e06-98ce-3ea11d44c114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:33.725547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8afb5b-cfb6-4e06-98ce-3ea11d44c114 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:33.728774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9ccbf39e-5221-48ff-a679-ef6bbf80582a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:33.732939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9ccbf39e-5221-48ff-a679-ef6bbf80582a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:33.743424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 776/1560] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:25:33.753263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8afb5b-cfb6-4e06-98ce-3ea11d44c114 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:33.765198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 785/1561] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:33.770127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97dc7f2-d77f-4ef3-9a83-5675c416488a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.770361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97dc7f2-d77f-4ef3-9a83-5675c416488a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.770652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97dc7f2-d77f-4ef3-9a83-5675c416488a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.778474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8afb5b-cfb6-4e06-98ce-3ea11d44c114 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:33.792246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015932c-d4f3-4edb-8769-7d2d8505869e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:33.792246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015932c-d4f3-4edb-8769-7d2d8505869e admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.792246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015932c-d4f3-4edb-8769-7d2d8505869e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.792246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015932c-d4f3-4edb-8769-7d2d8505869e admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.794821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015932c-d4f3-4edb-8769-7d2d8505869e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:33.802177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015932c-d4f3-4edb-8769-7d2d8505869e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:33.819859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e97dc7f2-d77f-4ef3-9a83-5675c416488a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:33.825938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e97dc7f2-d77f-4ef3-9a83-5675c416488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:33.858827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 786/1562] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:33.859040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 777/1563] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:33.859040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015932c-d4f3-4edb-8769-7d2d8505869e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:33.859226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97dc7f2-d77f-4ef3-9a83-5675c416488a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:33.876926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015932c-d4f3-4edb-8769-7d2d8505869e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:33.881077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97dc7f2-d77f-4ef3-9a83-5675c416488a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:33.882376 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4623de5-7cab-4a1b-b271-72d7d2f9c706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:33.883158 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4623de5-7cab-4a1b-b271-72d7d2f9c706 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.883501 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4623de5-7cab-4a1b-b271-72d7d2f9c706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.883786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4623de5-7cab-4a1b-b271-72d7d2f9c706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.885083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4623de5-7cab-4a1b-b271-72d7d2f9c706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:33.886982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4623de5-7cab-4a1b-b271-72d7d2f9c706 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:33.894376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6475ffd-76ca-42c7-86a0-0bae9556ceb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:33.896075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6475ffd-76ca-42c7-86a0-0bae9556ceb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.896918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6475ffd-76ca-42c7-86a0-0bae9556ceb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.897700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6475ffd-76ca-42c7-86a0-0bae9556ceb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.899503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6475ffd-76ca-42c7-86a0-0bae9556ceb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:33.904499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6475ffd-76ca-42c7-86a0-0bae9556ceb4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:33.922679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 778/1564] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:33.928993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6a339-fcd1-4267-a88a-26f56adeb1d7 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.929321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6a339-fcd1-4267-a88a-26f56adeb1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.929551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6a339-fcd1-4267-a88a-26f56adeb1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:33.958690 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5bd6a339-fcd1-4267-a88a-26f56adeb1d7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:33.959993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5bd6a339-fcd1-4267-a88a-26f56adeb1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:33.978820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 787/1565] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:33.984247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7ecb1-10d5-4bdd-84c1-1c6feca34138 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:33.985408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7ecb1-10d5-4bdd-84c1-1c6feca34138 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:33.986293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e7ecb1-10d5-4bdd-84c1-1c6feca34138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.041527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d4e7ecb1-10d5-4bdd-84c1-1c6feca34138 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:34.043623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d4e7ecb1-10d5-4bdd-84c1-1c6feca34138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:34.107387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 779/1566] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:34.112960 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6a339-fcd1-4267-a88a-26f56adeb1d7 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.130195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6a339-fcd1-4267-a88a-26f56adeb1d7 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.182771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 788/1567] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:34.184723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7ecb1-10d5-4bdd-84c1-1c6feca34138 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.199179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e7ecb1-10d5-4bdd-84c1-1c6feca34138 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.206057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bb9a7-5af0-4988-9a93-c1d59373721a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.206973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bb9a7-5af0-4988-9a93-c1d59373721a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.207315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bb9a7-5af0-4988-9a93-c1d59373721a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.207732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bb9a7-5af0-4988-9a93-c1d59373721a admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.208759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bb9a7-5af0-4988-9a93-c1d59373721a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.212383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ecb7a-36e4-4d96-abe1-4b0d22a538a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.213083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ecb7a-36e4-4d96-abe1-4b0d22a538a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.213377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ecb7a-36e4-4d96-abe1-4b0d22a538a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.213665 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ecb7a-36e4-4d96-abe1-4b0d22a538a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.214432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ecb7a-36e4-4d96-abe1-4b0d22a538a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:25:34.214787 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ecb7a-36e4-4d96-abe1-4b0d22a538a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.214932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bb9a7-5af0-4988-9a93-c1d59373721a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.216558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ecb7a-36e4-4d96-abe1-4b0d22a538a7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.236176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 789/1568] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:34.236176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bb9a7-5af0-4988-9a93-c1d59373721a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.236667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 780/1569] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:25:34.240654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9df876-0f04-4400-b042-4b7773f1af1d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.242938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9df876-0f04-4400-b042-4b7773f1af1d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.243289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9df876-0f04-4400-b042-4b7773f1af1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.249369 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bb9a7-5af0-4988-9a93-c1d59373721a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.254688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:34.254929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591e670-b304-4390-ae43-5a2df2d39239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.255496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591e670-b304-4390-ae43-5a2df2d39239 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.255785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591e670-b304-4390-ae43-5a2df2d39239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.256058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591e670-b304-4390-ae43-5a2df2d39239 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.257121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591e670-b304-4390-ae43-5a2df2d39239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.258887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f591e670-b304-4390-ae43-5a2df2d39239 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.280729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cf9df876-0f04-4400-b042-4b7773f1af1d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:34.283007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cf9df876-0f04-4400-b042-4b7773f1af1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:34.287461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 790/1570] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:34 2022] POST /identity/v3/users => generated 288 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:34.288578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591e670-b304-4390-ae43-5a2df2d39239 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.312743 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 781/1571] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:34.312743 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9df876-0f04-4400-b042-4b7773f1af1d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.313397 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591e670-b304-4390-ae43-5a2df2d39239 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.314458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26555545-2c22-411a-88b2-e8146f0cde80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.315900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26555545-2c22-411a-88b2-e8146f0cde80 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.316101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26555545-2c22-411a-88b2-e8146f0cde80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.316514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26555545-2c22-411a-88b2-e8146f0cde80 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.319191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26555545-2c22-411a-88b2-e8146f0cde80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.320210 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26555545-2c22-411a-88b2-e8146f0cde80 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.322384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9df876-0f04-4400-b042-4b7773f1af1d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.329766 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821746a-9630-435a-ae8c-f4b7ed480991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.330826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821746a-9630-435a-ae8c-f4b7ed480991 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.331801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821746a-9630-435a-ae8c-f4b7ed480991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.332367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821746a-9630-435a-ae8c-f4b7ed480991 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.334446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821746a-9630-435a-ae8c-f4b7ed480991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:25:34.334818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821746a-9630-435a-ae8c-f4b7ed480991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.339447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821746a-9630-435a-ae8c-f4b7ed480991 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.351010 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 791/1572] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:34 2022] POST /identity/v3/users => generated 293 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:34.355389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a112a28-8604-4135-8d0f-34720d0d7e9c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.355732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a112a28-8604-4135-8d0f-34720d0d7e9c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.356021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a112a28-8604-4135-8d0f-34720d0d7e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.361026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 782/1573] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:25:34.363114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821746a-9630-435a-ae8c-f4b7ed480991 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.384061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3a112a28-8604-4135-8d0f-34720d0d7e9c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:34.386058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3a112a28-8604-4135-8d0f-34720d0d7e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:34.392774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821746a-9630-435a-ae8c-f4b7ed480991 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.397101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 792/1574] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:34.397819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a112a28-8604-4135-8d0f-34720d0d7e9c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.407938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7713fc-c622-4f44-8d41-4f9d06423e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.409082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7713fc-c622-4f44-8d41-4f9d06423e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.409424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7713fc-c622-4f44-8d41-4f9d06423e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.409799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7713fc-c622-4f44-8d41-4f9d06423e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.410116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a112a28-8604-4135-8d0f-34720d0d7e9c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.411034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7713fc-c622-4f44-8d41-4f9d06423e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.412938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7713fc-c622-4f44-8d41-4f9d06423e18 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.416769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84de63e-64be-4d7c-8294-a2be0163c53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.417750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84de63e-64be-4d7c-8294-a2be0163c53a admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.418193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84de63e-64be-4d7c-8294-a2be0163c53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.418747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84de63e-64be-4d7c-8294-a2be0163c53a admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.419765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84de63e-64be-4d7c-8294-a2be0163c53a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.424645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84de63e-64be-4d7c-8294-a2be0163c53a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.429417 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:34.435021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 783/1575] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:34.435021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7713fc-c622-4f44-8d41-4f9d06423e18 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.463056 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7713fc-c622-4f44-8d41-4f9d06423e18 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.463699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 793/1576] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:34 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:34.465002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e319ff-ec66-4f5c-ab76-0b721d139674 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.465355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e319ff-ec66-4f5c-ab76-0b721d139674 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.465709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e319ff-ec66-4f5c-ab76-0b721d139674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.488306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-69e319ff-ec66-4f5c-ab76-0b721d139674 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:34.489578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-69e319ff-ec66-4f5c-ab76-0b721d139674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:34.594783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 794/1577] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:34.599666 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e319ff-ec66-4f5c-ab76-0b721d139674 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.614008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e319ff-ec66-4f5c-ab76-0b721d139674 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.616531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24534b1-fa3f-4295-bbdb-9d8b2fd5c792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.617701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24534b1-fa3f-4295-bbdb-9d8b2fd5c792 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.618402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24534b1-fa3f-4295-bbdb-9d8b2fd5c792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.618952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24534b1-fa3f-4295-bbdb-9d8b2fd5c792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.620012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344622c-6458-4d8c-b69e-4f479ab596e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.620958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344622c-6458-4d8c-b69e-4f479ab596e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.621330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24534b1-fa3f-4295-bbdb-9d8b2fd5c792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.621526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344622c-6458-4d8c-b69e-4f479ab596e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.621901 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8344622c-6458-4d8c-b69e-4f479ab596e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.623149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344622c-6458-4d8c-b69e-4f479ab596e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.624311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24534b1-fa3f-4295-bbdb-9d8b2fd5c792 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.625272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8344622c-6458-4d8c-b69e-4f479ab596e9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.643650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 795/1578] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:34.643650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344622c-6458-4d8c-b69e-4f479ab596e9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.651663 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 784/1579] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:34.651663 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24534b1-fa3f-4295-bbdb-9d8b2fd5c792 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.660115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8344622c-6458-4d8c-b69e-4f479ab596e9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.669396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fc060-5499-4fc8-9c87-b2217b3679e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.670041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fc060-5499-4fc8-9c87-b2217b3679e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.670543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fc060-5499-4fc8-9c87-b2217b3679e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.670944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2fc060-5499-4fc8-9c87-b2217b3679e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.671952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fc060-5499-4fc8-9c87-b2217b3679e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.675611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2fc060-5499-4fc8-9c87-b2217b3679e7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.676343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24534b1-fa3f-4295-bbdb-9d8b2fd5c792 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.689766 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687707cb-47d2-48ac-8aab-a042a2f6131d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.690964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687707cb-47d2-48ac-8aab-a042a2f6131d admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.691472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687707cb-47d2-48ac-8aab-a042a2f6131d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.691966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687707cb-47d2-48ac-8aab-a042a2f6131d admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.693360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687707cb-47d2-48ac-8aab-a042a2f6131d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:25:34.693906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687707cb-47d2-48ac-8aab-a042a2f6131d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.697494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687707cb-47d2-48ac-8aab-a042a2f6131d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.739470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 785/1580] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:25:34.739470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687707cb-47d2-48ac-8aab-a042a2f6131d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.739929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 796/1581] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:34 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:34.739929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fc060-5499-4fc8-9c87-b2217b3679e7 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.754989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687707cb-47d2-48ac-8aab-a042a2f6131d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.886160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2fc060-5499-4fc8-9c87-b2217b3679e7 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:34.893727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd03b9-fa75-44a2-a7ac-d85fb09b676f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.894758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd03b9-fa75-44a2-a7ac-d85fb09b676f admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.895049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd03b9-fa75-44a2-a7ac-d85fb09b676f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.895331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd03b9-fa75-44a2-a7ac-d85fb09b676f admin admin] PATH_INFO: `/v3/projects/375f46c4a1864eda97310c66a5239184/users/60794cf8fe974f239f79eee67509f205/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.896820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276decef-9ac0-4d34-b299-d9f88a15f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:34.897643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276decef-9ac0-4d34-b299-d9f88a15f0ea admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:34.898260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276decef-9ac0-4d34-b299-d9f88a15f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:34.898838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276decef-9ac0-4d34-b299-d9f88a15f0ea admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:34.899689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276decef-9ac0-4d34-b299-d9f88a15f0ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.903671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276decef-9ac0-4d34-b299-d9f88a15f0ea admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.915385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd03b9-fa75-44a2-a7ac-d85fb09b676f admin admin] RBAC: Authorizing `identity:create_grant(project_id=375f46c4a1864eda97310c66a5239184, user_id=60794cf8fe974f239f79eee67509f205, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:34.917051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-a1bd03b9-fa75-44a2-a7ac-d85fb09b676f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:25:34.917051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 20 08:25:34.917051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: Jul 20 08:25:34.919258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd03b9-fa75-44a2-a7ac-d85fb09b676f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:34.937566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 797/1582] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:34 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:34.939880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276decef-9ac0-4d34-b299-d9f88a15f0ea admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.943083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 786/1583] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:34 2022] PUT /identity/v3/projects/375f46c4a1864eda97310c66a5239184/users/60794cf8fe974f239f79eee67509f205/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:34.944398 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd03b9-fa75-44a2-a7ac-d85fb09b676f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:34.955479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276decef-9ac0-4d34-b299-d9f88a15f0ea admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.027973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:35.030672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:35.057283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebd36c-291c-4698-b710-3a061ca6e3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.058116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebd36c-291c-4698-b710-3a061ca6e3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.058709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebd36c-291c-4698-b710-3a061ca6e3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.059831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebd36c-291c-4698-b710-3a061ca6e3f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.060690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebd36c-291c-4698-b710-3a061ca6e3f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:25:35.061094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebd36c-291c-4698-b710-3a061ca6e3f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.063944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebd36c-291c-4698-b710-3a061ca6e3f0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.068680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd03b9-fa75-44a2-a7ac-d85fb09b676f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.074312 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb19dd-a49c-466e-aa97-d9836e7c7ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.075357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb19dd-a49c-466e-aa97-d9836e7c7ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.075830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb19dd-a49c-466e-aa97-d9836e7c7ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.076285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb19dd-a49c-466e-aa97-d9836e7c7ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.077584 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb19dd-a49c-466e-aa97-d9836e7c7ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.080085 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fb19dd-a49c-466e-aa97-d9836e7c7ee2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.081425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 798/1584] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:25:35.082618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebd36c-291c-4698-b710-3a061ca6e3f0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:35.096114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebd36c-291c-4698-b710-3a061ca6e3f0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.096629 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 787/1585] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:35.097907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb19dd-a49c-466e-aa97-d9836e7c7ee2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:35.105833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578bd48-e899-45da-9cc3-6dec0e9d80a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.106548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578bd48-e899-45da-9cc3-6dec0e9d80a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.106876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578bd48-e899-45da-9cc3-6dec0e9d80a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.107157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578bd48-e899-45da-9cc3-6dec0e9d80a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.108170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578bd48-e899-45da-9cc3-6dec0e9d80a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.111216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578bd48-e899-45da-9cc3-6dec0e9d80a6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.115918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb19dd-a49c-466e-aa97-d9836e7c7ee2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.122214 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132fd15-0f55-4787-a8cb-b53dd726a8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.123774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132fd15-0f55-4787-a8cb-b53dd726a8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.124548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132fd15-0f55-4787-a8cb-b53dd726a8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.128526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 799/1586] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:35.128526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578bd48-e899-45da-9cc3-6dec0e9d80a6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:35.129393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132fd15-0f55-4787-a8cb-b53dd726a8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.130709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132fd15-0f55-4787-a8cb-b53dd726a8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.133625 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132fd15-0f55-4787-a8cb-b53dd726a8f0 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.141960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578bd48-e899-45da-9cc3-6dec0e9d80a6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.154190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84ba79-1d5c-49ea-a12e-089c9e5a8b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.155029 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84ba79-1d5c-49ea-a12e-089c9e5a8b06 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.155330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84ba79-1d5c-49ea-a12e-089c9e5a8b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.156647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84ba79-1d5c-49ea-a12e-089c9e5a8b06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.159504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84ba79-1d5c-49ea-a12e-089c9e5a8b06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.162240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84ba79-1d5c-49ea-a12e-089c9e5a8b06 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.178161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 788/1587] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:35 2022] POST /identity/v3/users => generated 290 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:35.178161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132fd15-0f55-4787-a8cb-b53dd726a8f0 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:35.205850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132fd15-0f55-4787-a8cb-b53dd726a8f0 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.211516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 800/1588] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:35 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:35.213533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bca12-c07f-4403-a093-f7a1c283caaf None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.214076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bca12-c07f-4403-a093-f7a1c283caaf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.214524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374bca12-c07f-4403-a093-f7a1c283caaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.217403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf184f-4564-48f7-a6cc-41a9419249dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.219153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf184f-4564-48f7-a6cc-41a9419249dc admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.220689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf184f-4564-48f7-a6cc-41a9419249dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.221522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf184f-4564-48f7-a6cc-41a9419249dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.223972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf184f-4564-48f7-a6cc-41a9419249dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.227872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf184f-4564-48f7-a6cc-41a9419249dc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.255472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-374bca12-c07f-4403-a093-f7a1c283caaf None None] MFA Rules not processed for user `60794cf8fe974f239f79eee67509f205`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:35.257510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-374bca12-c07f-4403-a093-f7a1c283caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:35.267021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 789/1589] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:35.270961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf184f-4564-48f7-a6cc-41a9419249dc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:35.285285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf184f-4564-48f7-a6cc-41a9419249dc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.374289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493c60d-a3e5-4b99-8388-20f762b9964e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.375398 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493c60d-a3e5-4b99-8388-20f762b9964e admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.376477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493c60d-a3e5-4b99-8388-20f762b9964e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.377434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493c60d-a3e5-4b99-8388-20f762b9964e admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.381081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493c60d-a3e5-4b99-8388-20f762b9964e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.383911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493c60d-a3e5-4b99-8388-20f762b9964e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.439831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 790/1590] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:35 2022] POST /identity/v3/projects => generated 389 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:35.439831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493c60d-a3e5-4b99-8388-20f762b9964e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:35.445154 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 801/1591] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:35.446779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374bca12-c07f-4403-a093-f7a1c283caaf None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:35.461311 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374bca12-c07f-4403-a093-f7a1c283caaf None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.582560 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493c60d-a3e5-4b99-8388-20f762b9964e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.588411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eb6aa-07a9-4159-a6cd-209441b52ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.589342 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eb6aa-07a9-4159-a6cd-209441b52ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.589620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eb6aa-07a9-4159-a6cd-209441b52ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.589959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5eb6aa-07a9-4159-a6cd-209441b52ac8 admin admin] PATH_INFO: `/v3/projects/28f0e3883e9b4eca93f94a271e332ff7/users/12696013c1d64937a50db9fc36e548e9/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.604320 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c210ed-6f05-45bc-b1dc-a378bdd3a6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.608937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c210ed-6f05-45bc-b1dc-a378bdd3a6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.609924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c210ed-6f05-45bc-b1dc-a378bdd3a6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.612213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c210ed-6f05-45bc-b1dc-a378bdd3a6c9 admin admin] PATH_INFO: `/v3/projects/5b8be62fec184cf2872763f6e73b520d/users/4421872df5964fcd8269fb6abc45d943/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.617309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5eb6aa-07a9-4159-a6cd-209441b52ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f0e3883e9b4eca93f94a271e332ff7, user_id=12696013c1d64937a50db9fc36e548e9, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.621238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5eb6aa-07a9-4159-a6cd-209441b52ac8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.643080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 802/1592] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:35 2022] PUT /identity/v3/projects/28f0e3883e9b4eca93f94a271e332ff7/users/12696013c1d64937a50db9fc36e548e9/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:35.644429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eb6aa-07a9-4159-a6cd-209441b52ac8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:35.664410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c210ed-6f05-45bc-b1dc-a378bdd3a6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8be62fec184cf2872763f6e73b520d, user_id=4421872df5964fcd8269fb6abc45d943, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.667431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c210ed-6f05-45bc-b1dc-a378bdd3a6c9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.690139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 791/1593] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:35 2022] PUT /identity/v3/projects/5b8be62fec184cf2872763f6e73b520d/users/4421872df5964fcd8269fb6abc45d943/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:35.691386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c210ed-6f05-45bc-b1dc-a378bdd3a6c9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:35.696733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:35.731756 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:35.814893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c210ed-6f05-45bc-b1dc-a378bdd3a6c9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.818932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5eb6aa-07a9-4159-a6cd-209441b52ac8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.821288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7d35a-5085-4010-a51b-e996f55852e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.823007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7d35a-5085-4010-a51b-e996f55852e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.823416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7d35a-5085-4010-a51b-e996f55852e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.823815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7d35a-5085-4010-a51b-e996f55852e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.824804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7d35a-5085-4010-a51b-e996f55852e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.827824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979336e7-899a-4995-a9bf-0dc41b02df15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.828230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7d35a-5085-4010-a51b-e996f55852e8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.830219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979336e7-899a-4995-a9bf-0dc41b02df15 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.830827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979336e7-899a-4995-a9bf-0dc41b02df15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.831694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979336e7-899a-4995-a9bf-0dc41b02df15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.832758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979336e7-899a-4995-a9bf-0dc41b02df15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.834993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979336e7-899a-4995-a9bf-0dc41b02df15 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.855698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 803/1594] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:35.857564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979336e7-899a-4995-a9bf-0dc41b02df15 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:35.887522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979336e7-899a-4995-a9bf-0dc41b02df15 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.892584 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 792/1595] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:35 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:35.894846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7d35a-5085-4010-a51b-e996f55852e8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:35.898389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af32cf-945a-4c95-8ef9-65385895a13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.899733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af32cf-945a-4c95-8ef9-65385895a13e admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.900280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af32cf-945a-4c95-8ef9-65385895a13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.900902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af32cf-945a-4c95-8ef9-65385895a13e admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.902320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af32cf-945a-4c95-8ef9-65385895a13e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.905457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af32cf-945a-4c95-8ef9-65385895a13e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.912966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7d35a-5085-4010-a51b-e996f55852e8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:35.919835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cd750-6103-4b42-9296-08ea35c15080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:35.920702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd750-6103-4b42-9296-08ea35c15080 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.921150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd750-6103-4b42-9296-08ea35c15080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.922015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cd750-6103-4b42-9296-08ea35c15080 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.923119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cd750-6103-4b42-9296-08ea35c15080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:35.927370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 804/1596] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:35.928452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cd750-6103-4b42-9296-08ea35c15080 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:35.929273 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe24ba-51ff-4fd3-9f61-e4d194a4f390 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.929663 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe24ba-51ff-4fd3-9f61-e4d194a4f390 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.930264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe24ba-51ff-4fd3-9f61-e4d194a4f390 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.931756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 805/1597] 10.222.0.64 () {54 vars in 962 bytes} [Wed Jul 20 08:25:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:25:35.932912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a3a71-f378-406c-bb24-a39f2989f843 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.933271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a3a71-f378-406c-bb24-a39f2989f843 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.933658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a3a71-f378-406c-bb24-a39f2989f843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.966908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 793/1598] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:35 2022] POST /identity/v3/users => generated 295 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:35.969438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118248cf-9e53-4120-9227-30fbf9034329 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:35.969438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118248cf-9e53-4120-9227-30fbf9034329 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:35.969438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118248cf-9e53-4120-9227-30fbf9034329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:35.989975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-471a3a71-f378-406c-bb24-a39f2989f843 None None] MFA Rules not processed for user `12696013c1d64937a50db9fc36e548e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:35.992336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-471a3a71-f378-406c-bb24-a39f2989f843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:36.085265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-118248cf-9e53-4120-9227-30fbf9034329 None None] MFA Rules not processed for user `4421872df5964fcd8269fb6abc45d943`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:36.085265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-118248cf-9e53-4120-9227-30fbf9034329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:36.171904 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 806/1599] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:36.172803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a3a71-f378-406c-bb24-a39f2989f843 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:36.191217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a3a71-f378-406c-bb24-a39f2989f843 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:36.203625 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fcfc4-60c7-4b63-9279-9dcd81ce1cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:36.205633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fcfc4-60c7-4b63-9279-9dcd81ce1cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.206223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fcfc4-60c7-4b63-9279-9dcd81ce1cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.206759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fcfc4-60c7-4b63-9279-9dcd81ce1cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.209068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41fcfc4-60c7-4b63-9279-9dcd81ce1cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:36.216492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 794/1600] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:36.223165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41fcfc4-60c7-4b63-9279-9dcd81ce1cfa admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:36.224386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118248cf-9e53-4120-9227-30fbf9034329 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:36.237810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118248cf-9e53-4120-9227-30fbf9034329 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:36.240140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 807/1601] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:36.240902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fcfc4-60c7-4b63-9279-9dcd81ce1cfa admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:36.247489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfe782-62d7-4eeb-aeae-6756f7563680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:36.248490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe782-62d7-4eeb-aeae-6756f7563680 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.248734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe782-62d7-4eeb-aeae-6756f7563680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.249230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfe782-62d7-4eeb-aeae-6756f7563680 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.249956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfe782-62d7-4eeb-aeae-6756f7563680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:36.258015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fcfc4-60c7-4b63-9279-9dcd81ce1cfa admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:36.262101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfe782-62d7-4eeb-aeae-6756f7563680 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:36.264471 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6d304-0f9d-42b5-b544-287788c1a3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:36.265689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6d304-0f9d-42b5-b544-287788c1a3c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.266220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6d304-0f9d-42b5-b544-287788c1a3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.266671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6d304-0f9d-42b5-b544-287788c1a3c8 admin admin] PATH_INFO: `/v3/projects/60ee8039b1f4461c996165d5801b91f1/users/e6c2070c6d2a450c887b8ca6b97c4b38/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.292320 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 795/1602] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:36.294214 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c93d-f7c1-42ff-b058-ead354cb0f8c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.294629 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c93d-f7c1-42ff-b058-ead354cb0f8c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.295425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889c93d-f7c1-42ff-b058-ead354cb0f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.296216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6d304-0f9d-42b5-b544-287788c1a3c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60ee8039b1f4461c996165d5801b91f1, user_id=e6c2070c6d2a450c887b8ca6b97c4b38, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:36.301103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6d304-0f9d-42b5-b544-287788c1a3c8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:36.323071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 808/1603] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:36 2022] PUT /identity/v3/projects/60ee8039b1f4461c996165d5801b91f1/users/e6c2070c6d2a450c887b8ca6b97c4b38/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:36.324678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6d304-0f9d-42b5-b544-287788c1a3c8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:36.367360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0889c93d-f7c1-42ff-b058-ead354cb0f8c None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:36.369352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0889c93d-f7c1-42ff-b058-ead354cb0f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:36.435651 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:36.447139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6d304-0f9d-42b5-b544-287788c1a3c8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:36.463958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e5a9b-c783-48a4-8ab6-76dd61e67377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:36.465073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e5a9b-c783-48a4-8ab6-76dd61e67377 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.465439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e5a9b-c783-48a4-8ab6-76dd61e67377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.465794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e5a9b-c783-48a4-8ab6-76dd61e67377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.466714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e5a9b-c783-48a4-8ab6-76dd61e67377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:36.470097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e5a9b-c783-48a4-8ab6-76dd61e67377 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:36.472057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:36.499385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 809/1604] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:36.503607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e5a9b-c783-48a4-8ab6-76dd61e67377 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:36.507669 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 796/1605] 10.222.0.64 () {58 vars in 1059 bytes} [Wed Jul 20 08:25:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:36.508628 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c93d-f7c1-42ff-b058-ead354cb0f8c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:36.522467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889c93d-f7c1-42ff-b058-ead354cb0f8c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:36.525184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e5a9b-c783-48a4-8ab6-76dd61e67377 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:36.528145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679d8390-e0f5-4668-942a-ed093c40d523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:36.529154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d8390-e0f5-4668-942a-ed093c40d523 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.529442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d8390-e0f5-4668-942a-ed093c40d523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.529724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679d8390-e0f5-4668-942a-ed093c40d523 admin admin] PATH_INFO: `/v3/projects/1b481fa73344413d85fc2eb5c0376c9d/users/5bf1a059ddb34c2d8181712f53d592c2/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.535709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c01de-e9a6-4eac-a9f8-36e717f08111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:36.537387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c01de-e9a6-4eac-a9f8-36e717f08111 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.538034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c01de-e9a6-4eac-a9f8-36e717f08111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.538660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c01de-e9a6-4eac-a9f8-36e717f08111 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.540117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c01de-e9a6-4eac-a9f8-36e717f08111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:36.545232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c01de-e9a6-4eac-a9f8-36e717f08111 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:36.549505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d8390-e0f5-4668-942a-ed093c40d523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b481fa73344413d85fc2eb5c0376c9d, user_id=5bf1a059ddb34c2d8181712f53d592c2, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:36.552601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679d8390-e0f5-4668-942a-ed093c40d523 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:36.580156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 797/1606] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:36 2022] PUT /identity/v3/projects/1b481fa73344413d85fc2eb5c0376c9d/users/5bf1a059ddb34c2d8181712f53d592c2/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:36.587009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376d22c-484f-4f5a-a1de-7f83747458eb None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.587427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376d22c-484f-4f5a-a1de-7f83747458eb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.587725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376d22c-484f-4f5a-a1de-7f83747458eb None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.590837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 798/1607] 10.222.0.64 () {54 vars in 962 bytes} [Wed Jul 20 08:25:36 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:25:36.597296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 810/1608] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:36 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:36.597606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376d22c-484f-4f5a-a1de-7f83747458eb None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:36.597799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6cf26-4122-4943-ac14-050c20f1474c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.597799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6cf26-4122-4943-ac14-050c20f1474c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.597799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6cf26-4122-4943-ac14-050c20f1474c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.625814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3fc6cf26-4122-4943-ac14-050c20f1474c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:36.627700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3fc6cf26-4122-4943-ac14-050c20f1474c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:36.701337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376d22c-484f-4f5a-a1de-7f83747458eb None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:36.708528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814ea0b-ecba-4e74-9cab-fb35b637dc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:36.709466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814ea0b-ecba-4e74-9cab-fb35b637dc67 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.709937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814ea0b-ecba-4e74-9cab-fb35b637dc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.710399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814ea0b-ecba-4e74-9cab-fb35b637dc67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.711364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814ea0b-ecba-4e74-9cab-fb35b637dc67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:36.713739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814ea0b-ecba-4e74-9cab-fb35b637dc67 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:36.728588 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 799/1609] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:36.732654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814ea0b-ecba-4e74-9cab-fb35b637dc67 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:36.744882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814ea0b-ecba-4e74-9cab-fb35b637dc67 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:36.751450 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558abed-d246-4f2d-b327-c6b4ea026af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:36.752269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558abed-d246-4f2d-b327-c6b4ea026af5 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.752630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558abed-d246-4f2d-b327-c6b4ea026af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.753002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558abed-d246-4f2d-b327-c6b4ea026af5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.753854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558abed-d246-4f2d-b327-c6b4ea026af5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:36.755753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558abed-d246-4f2d-b327-c6b4ea026af5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:36.773517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 800/1610] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:36.773517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558abed-d246-4f2d-b327-c6b4ea026af5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:36.774320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 811/1611] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:36.777950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbff278-77c8-46bc-bc83-398722ca40f0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.778780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbff278-77c8-46bc-bc83-398722ca40f0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.779842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbff278-77c8-46bc-bc83-398722ca40f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.839439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2cbff278-77c8-46bc-bc83-398722ca40f0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:36.842041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2cbff278-77c8-46bc-bc83-398722ca40f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:36.857914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 812/1612] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:36.861296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d5842-f8aa-4a9f-a42f-310ffb617db6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.862177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d5842-f8aa-4a9f-a42f-310ffb617db6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.862684 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d5842-f8aa-4a9f-a42f-310ffb617db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:36.902052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f42d5842-f8aa-4a9f-a42f-310ffb617db6 None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:36.904512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f42d5842-f8aa-4a9f-a42f-310ffb617db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:36.934488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558abed-d246-4f2d-b327-c6b4ea026af5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:36.942792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca3ada-e387-4715-b7c9-908d61c39485 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:36.943541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca3ada-e387-4715-b7c9-908d61c39485 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:36.943873 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca3ada-e387-4715-b7c9-908d61c39485 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:36.944157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca3ada-e387-4715-b7c9-908d61c39485 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:37.036524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:37.082145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 813/1613] 10.222.0.64 () {58 vars in 1059 bytes} [Wed Jul 20 08:25:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:37.083918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d5842-f8aa-4a9f-a42f-310ffb617db6 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:37.097745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca3ada-e387-4715-b7c9-908d61c39485 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:37.099927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca3ada-e387-4715-b7c9-908d61c39485 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:37.106973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d5842-f8aa-4a9f-a42f-310ffb617db6 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:37.119386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410f353-0ab8-4e59-b312-dff5310e9dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:37.120693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410f353-0ab8-4e59-b312-dff5310e9dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:37.121181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410f353-0ab8-4e59-b312-dff5310e9dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:37.121656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410f353-0ab8-4e59-b312-dff5310e9dcc admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:37.123236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410f353-0ab8-4e59-b312-dff5310e9dcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:37.123902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 801/1614] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:37.124876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca3ada-e387-4715-b7c9-908d61c39485 service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:37.128577 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410f353-0ab8-4e59-b312-dff5310e9dcc admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:37.139111 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca3ada-e387-4715-b7c9-908d61c39485 service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:37.149126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44278598-3a98-4bfc-b7d6-8d2935524d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:37.151556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44278598-3a98-4bfc-b7d6-8d2935524d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:37.152078 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44278598-3a98-4bfc-b7d6-8d2935524d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:37.152648 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44278598-3a98-4bfc-b7d6-8d2935524d60 admin admin] PATH_INFO: `/v3/projects/fa4764ea3c4f4bf98ea56ee0798adfdc/users/b2326ff15f294e15806d8a564770b35e/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:37.178511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 814/1615] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:37 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:37.181681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410f353-0ab8-4e59-b312-dff5310e9dcc admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:37.193245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44278598-3a98-4bfc-b7d6-8d2935524d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4764ea3c4f4bf98ea56ee0798adfdc, user_id=b2326ff15f294e15806d8a564770b35e, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:37.196938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44278598-3a98-4bfc-b7d6-8d2935524d60 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:37.208742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410f353-0ab8-4e59-b312-dff5310e9dcc admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:37.238935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 802/1616] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:37 2022] PUT /identity/v3/projects/fa4764ea3c4f4bf98ea56ee0798adfdc/users/b2326ff15f294e15806d8a564770b35e/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:37.238935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44278598-3a98-4bfc-b7d6-8d2935524d60 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:37.330363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44278598-3a98-4bfc-b7d6-8d2935524d60 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:37.341261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c9cf1-35a0-49f7-af35-34e21e9ba3e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:37.343970 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c9cf1-35a0-49f7-af35-34e21e9ba3e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:37.345201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c9cf1-35a0-49f7-af35-34e21e9ba3e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:37.345778 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c9cf1-35a0-49f7-af35-34e21e9ba3e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:37.373314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:37.425194 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaade54-4d22-464b-b1d5-3aa50986eef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:37.426420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaade54-4d22-464b-b1d5-3aa50986eef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:37.428567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaade54-4d22-464b-b1d5-3aa50986eef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:37.429077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaade54-4d22-464b-b1d5-3aa50986eef3 admin admin] PATH_INFO: `/v3/projects/80faf6362e244bb992ffb6058ceeefda/users/fb5f0f7e59dd4c548a6ea72bb25f1ebb/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:37.468949 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaade54-4d22-464b-b1d5-3aa50986eef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80faf6362e244bb992ffb6058ceeefda, user_id=fb5f0f7e59dd4c548a6ea72bb25f1ebb, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:37.477032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaade54-4d22-464b-b1d5-3aa50986eef3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:37.518830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 815/1617] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:37 2022] PUT /identity/v3/projects/80faf6362e244bb992ffb6058ceeefda/users/fb5f0f7e59dd4c548a6ea72bb25f1ebb/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:37.521864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaade54-4d22-464b-b1d5-3aa50986eef3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:37.533617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c9cf1-35a0-49f7-af35-34e21e9ba3e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:37.535359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c9cf1-35a0-49f7-af35-34e21e9ba3e5 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:37.564304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 803/1618] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:37.565373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c9cf1-35a0-49f7-af35-34e21e9ba3e5 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:37.665511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaade54-4d22-464b-b1d5-3aa50986eef3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:37.673024 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e1cae-220d-41b3-b19b-d3608f72df32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:37.674994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e1cae-220d-41b3-b19b-d3608f72df32 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:37.675475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e1cae-220d-41b3-b19b-d3608f72df32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:37.676134 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e1cae-220d-41b3-b19b-d3608f72df32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:37.769611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:37.781488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c9cf1-35a0-49f7-af35-34e21e9ba3e5 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:37.913215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e1cae-220d-41b3-b19b-d3608f72df32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:37.913640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe8dd7-8a1a-4afa-a18d-fc4f77788036 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:37.914507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe8dd7-8a1a-4afa-a18d-fc4f77788036 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:37.915229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe8dd7-8a1a-4afa-a18d-fc4f77788036 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:37.915689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe8dd7-8a1a-4afa-a18d-fc4f77788036 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:37.919224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e1cae-220d-41b3-b19b-d3608f72df32 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:37.949792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 816/1619] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:37.952182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e1cae-220d-41b3-b19b-d3608f72df32 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:37.974547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e1cae-220d-41b3-b19b-d3608f72df32 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:37.983465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7448a6-2b88-4456-b664-afd193ab0238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:37.984632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7448a6-2b88-4456-b664-afd193ab0238 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:37.985217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7448a6-2b88-4456-b664-afd193ab0238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:37.985753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7448a6-2b88-4456-b664-afd193ab0238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:37.987456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7448a6-2b88-4456-b664-afd193ab0238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:37.990757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7448a6-2b88-4456-b664-afd193ab0238 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:38.016751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 817/1620] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:38.016751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888bf4d-ab70-4e2c-add2-356f1fd131f1 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:38.017085 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe8dd7-8a1a-4afa-a18d-fc4f77788036 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:38.017214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888bf4d-ab70-4e2c-add2-356f1fd131f1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:38.017485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888bf4d-ab70-4e2c-add2-356f1fd131f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:38.025086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe8dd7-8a1a-4afa-a18d-fc4f77788036 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:38.057189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 804/1621] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:38.064577 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0888bf4d-ab70-4e2c-add2-356f1fd131f1 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:38.067548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0888bf4d-ab70-4e2c-add2-356f1fd131f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:38.080367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d005-4e61-4f77-836b-748f77569b86 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:38.080685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d005-4e61-4f77-836b-748f77569b86 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:38.083225 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d005-4e61-4f77-836b-748f77569b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:38.089305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 818/1622] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:38.103647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da46fc3-b927-4d73-9b18-c06556335fff None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:38.105234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da46fc3-b927-4d73-9b18-c06556335fff None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:38.106319 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da46fc3-b927-4d73-9b18-c06556335fff None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:38.120205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 819/1623] 10.222.0.64 () {54 vars in 912 bytes} [Wed Jul 20 08:25:38 2022] GET /identity => generated 270 bytes in 30 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:25:38.122365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da46fc3-b927-4d73-9b18-c06556335fff None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:38.153633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da46fc3-b927-4d73-9b18-c06556335fff None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:38.159142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5b75d005-4e61-4f77-836b-748f77569b86 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:38.161400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5b75d005-4e61-4f77-836b-748f77569b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:38.178188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df8eb5-1983-43e7-9c3d-dd5b2d45744b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:38.183161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df8eb5-1983-43e7-9c3d-dd5b2d45744b admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:38.184199 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df8eb5-1983-43e7-9c3d-dd5b2d45744b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:38.185705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df8eb5-1983-43e7-9c3d-dd5b2d45744b admin admin] PATH_INFO: `/v3/projects/b3acd4d70ed94f3b9a473db433ccdbc1/users/5fac501d481248eab88d3871a116aaad/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:38.186532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 805/1624] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:38.192746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d005-4e61-4f77-836b-748f77569b86 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:38.234040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d005-4e61-4f77-836b-748f77569b86 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:38.238944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722e274-f3e8-4abe-82fa-19a9fd6f6480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:38.239925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722e274-f3e8-4abe-82fa-19a9fd6f6480 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:38.239925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722e274-f3e8-4abe-82fa-19a9fd6f6480 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:38.239925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722e274-f3e8-4abe-82fa-19a9fd6f6480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:38.270339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722e274-f3e8-4abe-82fa-19a9fd6f6480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:38.272051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722e274-f3e8-4abe-82fa-19a9fd6f6480 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:38.274122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df8eb5-1983-43e7-9c3d-dd5b2d45744b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3acd4d70ed94f3b9a473db433ccdbc1, user_id=5fac501d481248eab88d3871a116aaad, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:38.286230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10df8eb5-1983-43e7-9c3d-dd5b2d45744b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:38.311952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 806/1625] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:38.316759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6261214-e0ea-4bee-af0a-f7d46b4ef43c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:38.320552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6261214-e0ea-4bee-af0a-f7d46b4ef43c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:38.321093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6261214-e0ea-4bee-af0a-f7d46b4ef43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:38.369168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d6261214-e0ea-4bee-af0a-f7d46b4ef43c None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:38.370486 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d6261214-e0ea-4bee-af0a-f7d46b4ef43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:38.370867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 820/1626] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:38 2022] PUT /identity/v3/projects/b3acd4d70ed94f3b9a473db433ccdbc1/users/5fac501d481248eab88d3871a116aaad/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:38.378171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df8eb5-1983-43e7-9c3d-dd5b2d45744b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:38.526015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:38.620436 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 807/1627] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:25:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:38.624676 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6261214-e0ea-4bee-af0a-f7d46b4ef43c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:38.637346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df8eb5-1983-43e7-9c3d-dd5b2d45744b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:38.661193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b3336-a1e6-4536-aaa0-ee1e67f0c6ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:38.661673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:38.662333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b3336-a1e6-4536-aaa0-ee1e67f0c6ef service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:38.662881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b3336-a1e6-4536-aaa0-ee1e67f0c6ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:38.663389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b3336-a1e6-4536-aaa0-ee1e67f0c6ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:38.776333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6261214-e0ea-4bee-af0a-f7d46b4ef43c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:38.786500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9409d-5766-4403-ad6d-078034d1ec1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:38.787640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9409d-5766-4403-ad6d-078034d1ec1c admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:38.788154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9409d-5766-4403-ad6d-078034d1ec1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:38.788648 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f9409d-5766-4403-ad6d-078034d1ec1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:38.789644 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9409d-5766-4403-ad6d-078034d1ec1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:38.791682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f9409d-5766-4403-ad6d-078034d1ec1c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:38.820262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 808/1628] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:38.821687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9409d-5766-4403-ad6d-078034d1ec1c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:39.021153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f9409d-5766-4403-ad6d-078034d1ec1c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:39.034913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b3336-a1e6-4536-aaa0-ee1e67f0c6ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:39.040528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2061444-b204-499f-aa03-a65b3de17ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:39.041649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b3336-a1e6-4536-aaa0-ee1e67f0c6ef service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:39.042352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2061444-b204-499f-aa03-a65b3de17ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:39.042986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2061444-b204-499f-aa03-a65b3de17ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:39.044750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2061444-b204-499f-aa03-a65b3de17ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:39.090888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 821/1629] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:39.092506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b3336-a1e6-4536-aaa0-ee1e67f0c6ef service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:39.092935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2061444-b204-499f-aa03-a65b3de17ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:39.100389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2061444-b204-499f-aa03-a65b3de17ff6 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:39.142517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b3336-a1e6-4536-aaa0-ee1e67f0c6ef service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:39.154415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 809/1630] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:39.157385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cfa88-232c-4fb9-84c0-35c46489fc4d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:39.159037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cfa88-232c-4fb9-84c0-35c46489fc4d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:39.160280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cfa88-232c-4fb9-84c0-35c46489fc4d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:39.167943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 810/1631] 10.222.0.64 () {54 vars in 912 bytes} [Wed Jul 20 08:25:39 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:25:39.170954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0ec005-1fa8-41ea-bb22-61f101ec8b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:39.171927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ec005-1fa8-41ea-bb22-61f101ec8b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:39.174677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ec005-1fa8-41ea-bb22-61f101ec8b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:39.175038 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0ec005-1fa8-41ea-bb22-61f101ec8b05 admin admin] PATH_INFO: `/v3/projects/b3acd4d70ed94f3b9a473db433ccdbc1/users/5fac501d481248eab88d3871a116aaad/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:39.183498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ec005-1fa8-41ea-bb22-61f101ec8b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3acd4d70ed94f3b9a473db433ccdbc1, user_id=5fac501d481248eab88d3871a116aaad, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:39.185508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa52014-12dd-4dc6-bdcd-f3514117fb15 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:39.186257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa52014-12dd-4dc6-bdcd-f3514117fb15 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:39.186944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa52014-12dd-4dc6-bdcd-f3514117fb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:39.190022 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0ec005-1fa8-41ea-bb22-61f101ec8b05 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:39.259393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6aa52014-12dd-4dc6-bdcd-f3514117fb15 None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:39.268029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6aa52014-12dd-4dc6-bdcd-f3514117fb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:39.268782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 822/1632] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:39 2022] PUT /identity/v3/projects/b3acd4d70ed94f3b9a473db433ccdbc1/users/5fac501d481248eab88d3871a116aaad/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:39.291974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758555b-1b07-475d-aacc-4172cbe422cc None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:39.292514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758555b-1b07-475d-aacc-4172cbe422cc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:39.292946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758555b-1b07-475d-aacc-4172cbe422cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:39.409528 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e758555b-1b07-475d-aacc-4172cbe422cc None None] MFA Rules not processed for user `5fac501d481248eab88d3871a116aaad`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:39.417638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e758555b-1b07-475d-aacc-4172cbe422cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:39.551162 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 811/1633] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:25:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:39.584104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa52014-12dd-4dc6-bdcd-f3514117fb15 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:39.709405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 823/1634] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:39.765889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:39.796732 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa52014-12dd-4dc6-bdcd-f3514117fb15 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:39.805978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0f476-88eb-4311-8055-c7ed03c3cf4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:39.807906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f476-88eb-4311-8055-c7ed03c3cf4c service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:39.809474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f476-88eb-4311-8055-c7ed03c3cf4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:39.810304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f476-88eb-4311-8055-c7ed03c3cf4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:39.839832 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0f476-88eb-4311-8055-c7ed03c3cf4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:39.843807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0f476-88eb-4311-8055-c7ed03c3cf4c service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:39.873984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 812/1635] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:25:40.494686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758555b-1b07-475d-aacc-4172cbe422cc None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:40.514138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758555b-1b07-475d-aacc-4172cbe422cc None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:40.529475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca932f2-0712-43a9-b1b1-97ed8a50aa8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:40.530375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca932f2-0712-43a9-b1b1-97ed8a50aa8c service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:40.530795 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca932f2-0712-43a9-b1b1-97ed8a50aa8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:40.531174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca932f2-0712-43a9-b1b1-97ed8a50aa8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:40.669900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:40.685373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca932f2-0712-43a9-b1b1-97ed8a50aa8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:40.687966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca932f2-0712-43a9-b1b1-97ed8a50aa8c service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:40.702242 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 824/1636] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:44.240676 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0f476-88eb-4311-8055-c7ed03c3cf4c service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:44.435987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0f476-88eb-4311-8055-c7ed03c3cf4c service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:44.447104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:44.448315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:44.448766 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:44.449180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] PATH_INFO: `/v3/users/60794cf8fe974f239f79eee67509f205` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:44.450912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] RBAC: Authorizing `identity:delete_user(user_id=60794cf8fe974f239f79eee67509f205)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:44.452387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: WARNING py.warnings [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:25:44.452387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: warnings.warn(msg) Jul 20 08:25:44.452387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: Jul 20 08:25:44.454119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:44.579374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60794cf8fe974f239f79eee67509f205'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:44.598386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60794cf8fe974f239f79eee67509f205'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:44.612578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60794cf8fe974f239f79eee67509f205'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:44.647537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60794cf8fe974f239f79eee67509f205'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:44.648682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60794cf8fe974f239f79eee67509f205'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:44.658510 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 813/1637] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:25:44 2022] DELETE /identity/v3/users/60794cf8fe974f239f79eee67509f205 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:44.664961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a8a83-1899-42f0-be82-011b4ca7bd2d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:44.665951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a8a83-1899-42f0-be82-011b4ca7bd2d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:44.666488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a8a83-1899-42f0-be82-011b4ca7bd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:44.699032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d49a8a83-1899-42f0-be82-011b4ca7bd2d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:44.700985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d49a8a83-1899-42f0-be82-011b4ca7bd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:44.808275 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 825/1638] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:44.876019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:44.877869 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:45.060616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0e3a5-6e6d-4c93-b10c-566cc6d8e93a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:45.066464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7cfb07-5909-42af-a6ab-da15931976c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:45.068591 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cfb07-5909-42af-a6ab-da15931976c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:45.069176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cfb07-5909-42af-a6ab-da15931976c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:45.069698 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7cfb07-5909-42af-a6ab-da15931976c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:45.085636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7cfb07-5909-42af-a6ab-da15931976c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:45.088729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7cfb07-5909-42af-a6ab-da15931976c8 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:45.103660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 814/1639] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:25:47.757938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d4ac-556f-4ee5-a862-db4e248af117 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:47.758506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d4ac-556f-4ee5-a862-db4e248af117 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:47.766908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d4ac-556f-4ee5-a862-db4e248af117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:47.833674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5266d4ac-556f-4ee5-a862-db4e248af117 None None] MFA Rules not processed for user `bdc9e59d03334b72a8052e0f1112523f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:47.835141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5266d4ac-556f-4ee5-a862-db4e248af117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:47.951066 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 826/1640] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:47.963292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5975e6-5832-48aa-9033-8dc6e55d2366 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:47.963782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5975e6-5832-48aa-9033-8dc6e55d2366 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:47.964316 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5975e6-5832-48aa-9033-8dc6e55d2366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:48.050216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6b5975e6-5832-48aa-9033-8dc6e55d2366 None None] MFA Rules not processed for user `bdc9e59d03334b72a8052e0f1112523f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:48.055724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6b5975e6-5832-48aa-9033-8dc6e55d2366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:48.095355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 815/1641] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:48.122677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266d4ac-556f-4ee5-a862-db4e248af117 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:48.123129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5266d4ac-556f-4ee5-a862-db4e248af117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:48.194008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5975e6-5832-48aa-9033-8dc6e55d2366 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:48.194662 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6b5975e6-5832-48aa-9033-8dc6e55d2366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:48.213429 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:48.240424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5975e6-5832-48aa-9033-8dc6e55d2366 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:48.277854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:48.278910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:48.279204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:48.279491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] PATH_INFO: `/v3/projects/375f46c4a1864eda97310c66a5239184` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:48.288821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375f46c4a1864eda97310c66a5239184)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:48.308445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:48.438379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266d4ac-556f-4ee5-a862-db4e248af117 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:48.496449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375f46c4a1864eda97310c66a5239184', 'request_id': 'req-9a188894-8d67-4c99-8268-94a15408a245'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:48.508076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:48.529414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375f46c4a1864eda97310c66a5239184', 'request_id': 'req-9a188894-8d67-4c99-8268-94a15408a245'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:48.552054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375f46c4a1864eda97310c66a5239184', 'request_id': 'req-9a188894-8d67-4c99-8268-94a15408a245'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:48.600255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] The token cache is being invalidate because project 375f46c4a1864eda97310c66a5239184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:25:48.600789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:48.607379 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9a188894-8d67-4c99-8268-94a15408a245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:48.619166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 816/1642] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:25:48 2022] DELETE /identity/v3/projects/375f46c4a1864eda97310c66a5239184 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:48.638502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6167e-320a-409a-b09a-41ca7a46a2b4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:48.639255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6167e-320a-409a-b09a-41ca7a46a2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:48.639844 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6167e-320a-409a-b09a-41ca7a46a2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:48.708831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f3d6167e-320a-409a-b09a-41ca7a46a2b4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:48.710654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f3d6167e-320a-409a-b09a-41ca7a46a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:48.740149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe77af9-ff51-42a9-91a1-13ff1a2c128e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:48.741569 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe77af9-ff51-42a9-91a1-13ff1a2c128e service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:48.742267 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe77af9-ff51-42a9-91a1-13ff1a2c128e service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:48.742859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe77af9-ff51-42a9-91a1-13ff1a2c128e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:48.746411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ffe77af9-ff51-42a9-91a1-13ff1a2c128e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:48.878803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 817/1643] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:48.880209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd914f8-09e1-43a7-81db-62ac9e681cdb None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:48.881200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd914f8-09e1-43a7-81db-62ac9e681cdb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:48.881200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd914f8-09e1-43a7-81db-62ac9e681cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:48.959864 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ebd914f8-09e1-43a7-81db-62ac9e681cdb None None] MFA Rules not processed for user `fb5f0f7e59dd4c548a6ea72bb25f1ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:48.962408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ebd914f8-09e1-43a7-81db-62ac9e681cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:48.992020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe77af9-ff51-42a9-91a1-13ff1a2c128e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:48.995688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe77af9-ff51-42a9-91a1-13ff1a2c128e service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:49.029454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 827/1644] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 909 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:49.031361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a2a7-2a4b-413c-b254-e048ddfa8653 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:49.032883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a2a7-2a4b-413c-b254-e048ddfa8653 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:49.033615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246a2a7-2a4b-413c-b254-e048ddfa8653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:49.062143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d246a2a7-2a4b-413c-b254-e048ddfa8653 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:49.064782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d246a2a7-2a4b-413c-b254-e048ddfa8653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:49.075077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 828/1645] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:49.081948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246a2a7-2a4b-413c-b254-e048ddfa8653 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:49.096305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246a2a7-2a4b-413c-b254-e048ddfa8653 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:49.102741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c60529-27cf-4c24-9389-2e96b6e84577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:49.104937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60529-27cf-4c24-9389-2e96b6e84577 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:49.105451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60529-27cf-4c24-9389-2e96b6e84577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:49.105931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c60529-27cf-4c24-9389-2e96b6e84577 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:49.107161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c60529-27cf-4c24-9389-2e96b6e84577 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:25:49.107710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c60529-27cf-4c24-9389-2e96b6e84577 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:49.110395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c60529-27cf-4c24-9389-2e96b6e84577 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:49.124125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 818/1646] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:49.149247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 829/1647] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:25:49.157879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd914f8-09e1-43a7-81db-62ac9e681cdb None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:49.181201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c60529-27cf-4c24-9389-2e96b6e84577 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:49.184713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f7c60529-27cf-4c24-9389-2e96b6e84577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:49.217969 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd914f8-09e1-43a7-81db-62ac9e681cdb None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:49.242765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34f15b-40d9-4048-9378-a16cdc600acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:49.244525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f15b-40d9-4048-9378-a16cdc600acc admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:49.244525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f15b-40d9-4048-9378-a16cdc600acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:49.244932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34f15b-40d9-4048-9378-a16cdc600acc admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:49.246361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34f15b-40d9-4048-9378-a16cdc600acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:49.252542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c60529-27cf-4c24-9389-2e96b6e84577 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:49.261353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34f15b-40d9-4048-9378-a16cdc600acc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:49.285563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44322b-7964-43f0-b4bf-c2a3185ed89f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:49.286743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44322b-7964-43f0-b4bf-c2a3185ed89f service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:49.287262 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44322b-7964-43f0-b4bf-c2a3185ed89f service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:49.287776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44322b-7964-43f0-b4bf-c2a3185ed89f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:49.333391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:49.340686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44322b-7964-43f0-b4bf-c2a3185ed89f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:49.349017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44322b-7964-43f0-b4bf-c2a3185ed89f service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:49.377209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 819/1648] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:49 2022] POST /identity/v3/projects => generated 373 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:49.378655 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34f15b-40d9-4048-9378-a16cdc600acc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:49.382817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 830/1649] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:49.747213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34f15b-40d9-4048-9378-a16cdc600acc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:49.759654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6a83c-84ee-48f3-aadb-0e6727bbd3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:49.763871 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6a83c-84ee-48f3-aadb-0e6727bbd3e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:49.766166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6a83c-84ee-48f3-aadb-0e6727bbd3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:49.767571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6a83c-84ee-48f3-aadb-0e6727bbd3e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:49.769884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6a83c-84ee-48f3-aadb-0e6727bbd3e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:49.774941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b6a83c-84ee-48f3-aadb-0e6727bbd3e5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:49.876894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 820/1650] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:49 2022] POST /identity/v3/users => generated 287 bytes in 499 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:49.877592 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44322b-7964-43f0-b4bf-c2a3185ed89f service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:49.903590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44322b-7964-43f0-b4bf-c2a3185ed89f service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:49.924461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a30b01-b376-4aab-af72-692268206913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:49.925454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a30b01-b376-4aab-af72-692268206913 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:49.925982 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a30b01-b376-4aab-af72-692268206913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:49.926469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a30b01-b376-4aab-af72-692268206913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:49.927959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a30b01-b376-4aab-af72-692268206913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:49.930068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a30b01-b376-4aab-af72-692268206913 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:49.951049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 831/1651] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:49.959025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6a83c-84ee-48f3-aadb-0e6727bbd3e5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:49.992275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b6a83c-84ee-48f3-aadb-0e6727bbd3e5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:50.026002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563d4bc7-5531-4885-b92f-47ec0b365553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:50.027055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d4bc7-5531-4885-b92f-47ec0b365553 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:50.028106 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d4bc7-5531-4885-b92f-47ec0b365553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:50.028624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563d4bc7-5531-4885-b92f-47ec0b365553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:50.030383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d4bc7-5531-4885-b92f-47ec0b365553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:50.034659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563d4bc7-5531-4885-b92f-47ec0b365553 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:50.069041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 821/1652] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:50.076939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a30b01-b376-4aab-af72-692268206913 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:50.103603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a30b01-b376-4aab-af72-692268206913 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:50.114537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86104a-204d-44cc-9322-17df9118ad85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:50.118337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86104a-204d-44cc-9322-17df9118ad85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:50.119456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86104a-204d-44cc-9322-17df9118ad85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:50.123993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86104a-204d-44cc-9322-17df9118ad85 admin admin] PATH_INFO: `/v3/projects/a683127be5754f4e9b144a505638eeaf/users/c4e062536b544daf9fa869bff6eb28ba/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:50.203230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86104a-204d-44cc-9322-17df9118ad85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a683127be5754f4e9b144a505638eeaf, user_id=c4e062536b544daf9fa869bff6eb28ba, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:50.209011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86104a-204d-44cc-9322-17df9118ad85 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:50.253136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 832/1653] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:50 2022] PUT /identity/v3/projects/a683127be5754f4e9b144a505638eeaf/users/c4e062536b544daf9fa869bff6eb28ba/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:50.255592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899001-da81-438f-a168-e9c243443381 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:50.256424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899001-da81-438f-a168-e9c243443381 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:50.257534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0899001-da81-438f-a168-e9c243443381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:50.358367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c0899001-da81-438f-a168-e9c243443381 None None] MFA Rules not processed for user `c4e062536b544daf9fa869bff6eb28ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:50.369589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c0899001-da81-438f-a168-e9c243443381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:50.564039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946df63-96c0-4fb0-b5ee-6cae17feab17 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:50.564511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946df63-96c0-4fb0-b5ee-6cae17feab17 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:50.565448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946df63-96c0-4fb0-b5ee-6cae17feab17 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:50.567380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 833/1654] 10.222.0.64 () {54 vars in 912 bytes} [Wed Jul 20 08:25:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:25:50.575167 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3ae70-fc4f-4050-8c51-4d7aeaea5390 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:50.575735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3ae70-fc4f-4050-8c51-4d7aeaea5390 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:50.576018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3ae70-fc4f-4050-8c51-4d7aeaea5390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:50.717574 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 822/1655] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:50.731433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0899001-da81-438f-a168-e9c243443381 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:50.734260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-dfe3ae70-fc4f-4050-8c51-4d7aeaea5390 None None] MFA Rules not processed for user `a3b03383554346138b98becb2828851a`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:50.736444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dfe3ae70-fc4f-4050-8c51-4d7aeaea5390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:50.873832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:50.935800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:50.963113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 834/1656] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:25:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 20 08:25:50.992901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0899001-da81-438f-a168-e9c243443381 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:51.005116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3ae70-fc4f-4050-8c51-4d7aeaea5390 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:51.006969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dfe3ae70-fc4f-4050-8c51-4d7aeaea5390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:51.023796 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3ae70-fc4f-4050-8c51-4d7aeaea5390 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:51.032226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a5bf-e687-44ef-b449-ceff2db90e02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:51.033205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a5bf-e687-44ef-b449-ceff2db90e02 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:51.033495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a5bf-e687-44ef-b449-ceff2db90e02 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:51.033780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7a5bf-e687-44ef-b449-ceff2db90e02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:51.046444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a52d75-93a8-4600-babb-15ecde2222d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:51.047184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a52d75-93a8-4600-babb-15ecde2222d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:51.047697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a52d75-93a8-4600-babb-15ecde2222d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:51.048150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a52d75-93a8-4600-babb-15ecde2222d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:51.080965 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7a5bf-e687-44ef-b449-ceff2db90e02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:51.083240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7a5bf-e687-44ef-b449-ceff2db90e02 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:51.096396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a52d75-93a8-4600-babb-15ecde2222d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:51.097889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a52d75-93a8-4600-babb-15ecde2222d5 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:51.111142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 823/1657] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:51.131721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 835/1658] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:25:51.516208 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a5bf-e687-44ef-b449-ceff2db90e02 service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:51.749154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7a5bf-e687-44ef-b449-ceff2db90e02 service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:51.755780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833c285-e98e-4985-b3de-d8e89fe2a0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:51.757295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833c285-e98e-4985-b3de-d8e89fe2a0a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:51.757755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833c285-e98e-4985-b3de-d8e89fe2a0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:51.758284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833c285-e98e-4985-b3de-d8e89fe2a0a5 admin admin] PATH_INFO: `/v3/users/c4e062536b544daf9fa869bff6eb28ba` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:51.759871 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833c285-e98e-4985-b3de-d8e89fe2a0a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4e062536b544daf9fa869bff6eb28ba)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:51.762301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833c285-e98e-4985-b3de-d8e89fe2a0a5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:51.862875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a833c285-e98e-4985-b3de-d8e89fe2a0a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4e062536b544daf9fa869bff6eb28ba'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:51.874912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a833c285-e98e-4985-b3de-d8e89fe2a0a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4e062536b544daf9fa869bff6eb28ba'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:51.889547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a833c285-e98e-4985-b3de-d8e89fe2a0a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4e062536b544daf9fa869bff6eb28ba'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:51.909182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a833c285-e98e-4985-b3de-d8e89fe2a0a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4e062536b544daf9fa869bff6eb28ba'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:51.910317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a833c285-e98e-4985-b3de-d8e89fe2a0a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4e062536b544daf9fa869bff6eb28ba'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:51.921116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 824/1659] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:25:51 2022] DELETE /identity/v3/users/c4e062536b544daf9fa869bff6eb28ba => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:51.930443 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fedf3-97bf-4b58-b270-ae51ca49a31d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:51.931026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fedf3-97bf-4b58-b270-ae51ca49a31d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:51.931507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fedf3-97bf-4b58-b270-ae51ca49a31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:52.015892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a08fedf3-97bf-4b58-b270-ae51ca49a31d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:52.018848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a08fedf3-97bf-4b58-b270-ae51ca49a31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:52.354157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 836/1660] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:52.363668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60dd7-e06b-43e6-aa0e-18d1934d6230 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:52.363947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60dd7-e06b-43e6-aa0e-18d1934d6230 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:52.364296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e60dd7-e06b-43e6-aa0e-18d1934d6230 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:52.366181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 825/1661] 10.222.0.64 () {54 vars in 961 bytes} [Wed Jul 20 08:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:25:52.374308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8829-e0c5-4590-a1fa-ce471bb98048 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:52.374749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8829-e0c5-4590-a1fa-ce471bb98048 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:52.374986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea8829-e0c5-4590-a1fa-ce471bb98048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:52.404949 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-54ea8829-e0c5-4590-a1fa-ce471bb98048 None None] MFA Rules not processed for user `928054a577d14522bd0f00417f4f8c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:52.406350 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-54ea8829-e0c5-4590-a1fa-ce471bb98048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:52.545552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e60dd7-e06b-43e6-aa0e-18d1934d6230 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:52.548611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f3e60dd7-e06b-43e6-aa0e-18d1934d6230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:52.551397 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 837/1662] 10.222.0.64 () {58 vars in 1058 bytes} [Wed Jul 20 08:25:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:25:52.552510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea8829-e0c5-4590-a1fa-ce471bb98048 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:52.579865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea8829-e0c5-4590-a1fa-ce471bb98048 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:52.600052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3036d7-027a-44ae-9dd6-2b756b9c052c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:52.600913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3036d7-027a-44ae-9dd6-2b756b9c052c service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:52.601332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3036d7-027a-44ae-9dd6-2b756b9c052c service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:52.601688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3036d7-027a-44ae-9dd6-2b756b9c052c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:52.605191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bf3036d7-027a-44ae-9dd6-2b756b9c052c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:52.635353 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3036d7-027a-44ae-9dd6-2b756b9c052c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:52.637350 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3036d7-027a-44ae-9dd6-2b756b9c052c service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:52.691935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 838/1663] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:52.744738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e60dd7-e06b-43e6-aa0e-18d1934d6230 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:52.756877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3bb74-0f7b-4061-8c1d-65752b726af8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:52.757715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3bb74-0f7b-4061-8c1d-65752b726af8 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:52.758121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3bb74-0f7b-4061-8c1d-65752b726af8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:52.758406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3bb74-0f7b-4061-8c1d-65752b726af8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:52.786309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3bb74-0f7b-4061-8c1d-65752b726af8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:52.790807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb3bb74-0f7b-4061-8c1d-65752b726af8 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:52.815237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 826/1664] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:52.911623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f62b8-9170-4e93-9263-518d6d404fd3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:52.912000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f62b8-9170-4e93-9263-518d6d404fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:52.912384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f62b8-9170-4e93-9263-518d6d404fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:52.951567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d47f62b8-9170-4e93-9263-518d6d404fd3 None None] MFA Rules not processed for user `bdc9e59d03334b72a8052e0f1112523f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:52.952844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d47f62b8-9170-4e93-9263-518d6d404fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:53.112478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:53.130202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 839/1665] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:55.131478 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3bb74-0f7b-4061-8c1d-65752b726af8 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:55.134231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6fb3bb74-0f7b-4061-8c1d-65752b726af8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:55.189953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb3bb74-0f7b-4061-8c1d-65752b726af8 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:55.210426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:55.211386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:55.211934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:55.212296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] PATH_INFO: `/v3/projects/a683127be5754f4e9b144a505638eeaf` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:55.216995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a683127be5754f4e9b144a505638eeaf)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:55.219142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:55.223139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:55.333870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f62b8-9170-4e93-9263-518d6d404fd3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:55.336381 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d47f62b8-9170-4e93-9263-518d6d404fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:55.381626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a683127be5754f4e9b144a505638eeaf', 'request_id': 'req-6ef041bf-889a-45f3-9b21-bc96e396ed08'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:55.394241 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a683127be5754f4e9b144a505638eeaf', 'request_id': 'req-6ef041bf-889a-45f3-9b21-bc96e396ed08'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:55.409059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a683127be5754f4e9b144a505638eeaf', 'request_id': 'req-6ef041bf-889a-45f3-9b21-bc96e396ed08'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:55.451619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] The token cache is being invalidate because project a683127be5754f4e9b144a505638eeaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:25:55.452178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:55.454197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6ef041bf-889a-45f3-9b21-bc96e396ed08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:55.479196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 827/1666] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:25:55 2022] DELETE /identity/v3/projects/a683127be5754f4e9b144a505638eeaf => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:55.481827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfae80-cffa-400b-accf-d84a27510c33 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:55.482870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfae80-cffa-400b-accf-d84a27510c33 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:55.482870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfae80-cffa-400b-accf-d84a27510c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:55.529276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f62b8-9170-4e93-9263-518d6d404fd3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:55.620600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d91991-0f31-4c19-8324-1fe49131a901 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:55.621720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d91991-0f31-4c19-8324-1fe49131a901 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:55.622226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d91991-0f31-4c19-8324-1fe49131a901 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:55.622922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d91991-0f31-4c19-8324-1fe49131a901 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:55.627012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d6d91991-0f31-4c19-8324-1fe49131a901 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:55.642672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-62cfae80-cffa-400b-accf-d84a27510c33 None None] MFA Rules not processed for user `fb5f0f7e59dd4c548a6ea72bb25f1ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:55.645049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-62cfae80-cffa-400b-accf-d84a27510c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:55.725209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d91991-0f31-4c19-8324-1fe49131a901 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:55.727279 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d91991-0f31-4c19-8324-1fe49131a901 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:55.741073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 840/1667] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:25:55.742715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5e802-fd28-4f03-91b3-4740d6004372 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:55.743283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5e802-fd28-4f03-91b3-4740d6004372 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:55.743770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5e802-fd28-4f03-91b3-4740d6004372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:55.768709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-86c5e802-fd28-4f03-91b3-4740d6004372 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:55.770646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-86c5e802-fd28-4f03-91b3-4740d6004372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:55.856128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 841/1668] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:55.871729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad52ea-5651-4ef2-a1ca-77aff8bc6719 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:55.872185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad52ea-5651-4ef2-a1ca-77aff8bc6719 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:55.872564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad52ea-5651-4ef2-a1ca-77aff8bc6719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:55.909033 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:55.925337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-fbad52ea-5651-4ef2-a1ca-77aff8bc6719 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:55.930525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fbad52ea-5651-4ef2-a1ca-77aff8bc6719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:55.952203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 842/1669] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:25:55.958493 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad52ea-5651-4ef2-a1ca-77aff8bc6719 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:55.962243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 828/1670] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:55.988968 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad52ea-5651-4ef2-a1ca-77aff8bc6719 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:56.010696 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d39141-a7be-4a88-ad9e-b5b1c940b903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:56.012806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d39141-a7be-4a88-ad9e-b5b1c940b903 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:56.015651 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d39141-a7be-4a88-ad9e-b5b1c940b903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:56.017771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d39141-a7be-4a88-ad9e-b5b1c940b903 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:56.021231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d39141-a7be-4a88-ad9e-b5b1c940b903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:25:56.021592 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d39141-a7be-4a88-ad9e-b5b1c940b903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:56.024573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d39141-a7be-4a88-ad9e-b5b1c940b903 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:56.067164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 843/1671] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:25:56.074119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfae80-cffa-400b-accf-d84a27510c33 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:56.113786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfae80-cffa-400b-accf-d84a27510c33 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:56.135593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff0e44-485e-48f8-b67b-de36e133998f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:56.136705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff0e44-485e-48f8-b67b-de36e133998f admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:56.137307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff0e44-485e-48f8-b67b-de36e133998f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:56.137677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff0e44-485e-48f8-b67b-de36e133998f admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:56.138991 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff0e44-485e-48f8-b67b-de36e133998f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:56.143617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bff0e44-485e-48f8-b67b-de36e133998f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:56.303261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 829/1672] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:56 2022] POST /identity/v3/projects => generated 381 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:56.304427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d39141-a7be-4a88-ad9e-b5b1c940b903 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:56.325822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d39141-a7be-4a88-ad9e-b5b1c940b903 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:56.625110 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d15a03-1f51-4f0e-aa3a-c0e1ebaab828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:56.627682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d15a03-1f51-4f0e-aa3a-c0e1ebaab828 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:56.628443 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d15a03-1f51-4f0e-aa3a-c0e1ebaab828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:56.629196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d15a03-1f51-4f0e-aa3a-c0e1ebaab828 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:56.630767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d15a03-1f51-4f0e-aa3a-c0e1ebaab828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:56.639702 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d15a03-1f51-4f0e-aa3a-c0e1ebaab828 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:56.731605 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 844/1673] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:56 2022] POST /identity/v3/users => generated 291 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:56.748605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff0e44-485e-48f8-b67b-de36e133998f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:56.804765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff0e44-485e-48f8-b67b-de36e133998f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:56.829200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eef969-7253-4c76-96f2-153ab0cf199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:56.830376 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eef969-7253-4c76-96f2-153ab0cf199c admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:56.830937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eef969-7253-4c76-96f2-153ab0cf199c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:56.831679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eef969-7253-4c76-96f2-153ab0cf199c admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:56.840300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eef969-7253-4c76-96f2-153ab0cf199c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:56.843921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eef969-7253-4c76-96f2-153ab0cf199c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:56.890103 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 830/1674] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:56.915662 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d15a03-1f51-4f0e-aa3a-c0e1ebaab828 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:56.953377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d15a03-1f51-4f0e-aa3a-c0e1ebaab828 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:56.974888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418af41-2d78-4b8b-b2bd-744a31fd057e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:56.976127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418af41-2d78-4b8b-b2bd-744a31fd057e admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:56.976766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418af41-2d78-4b8b-b2bd-744a31fd057e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:56.977242 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418af41-2d78-4b8b-b2bd-744a31fd057e admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:56.978220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418af41-2d78-4b8b-b2bd-744a31fd057e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:56.983486 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418af41-2d78-4b8b-b2bd-744a31fd057e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:57.023938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eef969-7253-4c76-96f2-153ab0cf199c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:57.025007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 845/1675] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:57.052949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eef969-7253-4c76-96f2-153ab0cf199c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:57.069965 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418af41-2d78-4b8b-b2bd-744a31fd057e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:57.073512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b418af41-2d78-4b8b-b2bd-744a31fd057e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:57.098538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a604677-962a-4ab2-9c31-c70753ac8bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:57.098538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a604677-962a-4ab2-9c31-c70753ac8bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:57.098538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a604677-962a-4ab2-9c31-c70753ac8bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:57.098538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a604677-962a-4ab2-9c31-c70753ac8bbf admin admin] PATH_INFO: `/v3/projects/385a53799ce044f898f742ebd3dd79da/users/bb1b6336acb2484e9642db8bf7b21efd/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:57.117362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418af41-2d78-4b8b-b2bd-744a31fd057e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:57.149716 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a0b33d-40d3-455d-84de-7bdd300514ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:57.155831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0b33d-40d3-455d-84de-7bdd300514ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:57.157039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0b33d-40d3-455d-84de-7bdd300514ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:57.157039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a0b33d-40d3-455d-84de-7bdd300514ae admin admin] PATH_INFO: `/v3/users/4421872df5964fcd8269fb6abc45d943` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:57.163765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0b33d-40d3-455d-84de-7bdd300514ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=4421872df5964fcd8269fb6abc45d943)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:57.178829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a604677-962a-4ab2-9c31-c70753ac8bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=385a53799ce044f898f742ebd3dd79da, user_id=bb1b6336acb2484e9642db8bf7b21efd, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:57.207997 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a604677-962a-4ab2-9c31-c70753ac8bbf admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:57.223441 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a0b33d-40d3-455d-84de-7bdd300514ae admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:57.299292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 831/1676] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:57 2022] PUT /identity/v3/projects/385a53799ce044f898f742ebd3dd79da/users/bb1b6336acb2484e9642db8bf7b21efd/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:25:57.320239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a4287-78b2-445f-b4dc-2d77c2ca755c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:57.322151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a4287-78b2-445f-b4dc-2d77c2ca755c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:57.322676 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a4287-78b2-445f-b4dc-2d77c2ca755c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:57.420119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-333a4287-78b2-445f-b4dc-2d77c2ca755c None None] MFA Rules not processed for user `bb1b6336acb2484e9642db8bf7b21efd`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:57.424954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-333a4287-78b2-445f-b4dc-2d77c2ca755c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:57.455539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b0a0b33d-40d3-455d-84de-7bdd300514ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4421872df5964fcd8269fb6abc45d943'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:57.519497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b0a0b33d-40d3-455d-84de-7bdd300514ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4421872df5964fcd8269fb6abc45d943'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:57.520429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b0a0b33d-40d3-455d-84de-7bdd300514ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4421872df5964fcd8269fb6abc45d943'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:57.537958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b0a0b33d-40d3-455d-84de-7bdd300514ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4421872df5964fcd8269fb6abc45d943'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:57.548267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:57.561564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b0a0b33d-40d3-455d-84de-7bdd300514ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4421872df5964fcd8269fb6abc45d943'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:25:57.595189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 846/1677] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:25:57 2022] DELETE /identity/v3/users/4421872df5964fcd8269fb6abc45d943 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:25:57.601888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c27c3-ffed-425a-96c9-3656f1eca8d8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:57.604071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c27c3-ffed-425a-96c9-3656f1eca8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:57.604524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69c27c3-ffed-425a-96c9-3656f1eca8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:57.674916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c69c27c3-ffed-425a-96c9-3656f1eca8d8 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:57.676444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c69c27c3-ffed-425a-96c9-3656f1eca8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:57.706733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 832/1678] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:57.716865 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ec8a5-a16b-4c25-8e00-19f7a5f1c8ac None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:57.719816 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ec8a5-a16b-4c25-8e00-19f7a5f1c8ac None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:57.720299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ec8a5-a16b-4c25-8e00-19f7a5f1c8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:57.831274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:57.920599 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-687ec8a5-a16b-4c25-8e00-19f7a5f1c8ac None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:25:57.921885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-687ec8a5-a16b-4c25-8e00-19f7a5f1c8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:58.019860 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 847/1679] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:58.021920 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c27c3-ffed-425a-96c9-3656f1eca8d8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:58.056968 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c69c27c3-ffed-425a-96c9-3656f1eca8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:58.119550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69c27c3-ffed-425a-96c9-3656f1eca8d8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:58.144936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a6f57-b3e6-4fe3-91bb-8a7dc9e1a6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:58.146195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6f57-b3e6-4fe3-91bb-8a7dc9e1a6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:58.146930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6f57-b3e6-4fe3-91bb-8a7dc9e1a6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:58.148125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a6f57-b3e6-4fe3-91bb-8a7dc9e1a6b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:58.149812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a6f57-b3e6-4fe3-91bb-8a7dc9e1a6b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:58.153955 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a6f57-b3e6-4fe3-91bb-8a7dc9e1a6b6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:58.294843 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 848/1680] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:25:58 2022] POST /identity/v3/projects => generated 381 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:58.301604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a6f57-b3e6-4fe3-91bb-8a7dc9e1a6b6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:58.310107 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d32a6f57-b3e6-4fe3-91bb-8a7dc9e1a6b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:58.317461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 833/1681] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:25:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 610 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:25:58.326060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ec8a5-a16b-4c25-8e00-19f7a5f1c8ac None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:58.402412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ec8a5-a16b-4c25-8e00-19f7a5f1c8ac None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:58.840967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a6f57-b3e6-4fe3-91bb-8a7dc9e1a6b6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:58.876319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0c65b-7b5c-4ab1-868e-fb5c2f691d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:58.883836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c65b-7b5c-4ab1-868e-fb5c2f691d88 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:58.884990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa9778-4665-454e-9f3a-e59a39fdee90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:58.885333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c65b-7b5c-4ab1-868e-fb5c2f691d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:58.885755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c65b-7b5c-4ab1-868e-fb5c2f691d88 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:58.887289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0c65b-7b5c-4ab1-868e-fb5c2f691d88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:58.888852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa9778-4665-454e-9f3a-e59a39fdee90 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:58.894657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa9778-4665-454e-9f3a-e59a39fdee90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:58.899865 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0c65b-7b5c-4ab1-868e-fb5c2f691d88 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:58.900837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa9778-4665-454e-9f3a-e59a39fdee90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:58.956160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa9778-4665-454e-9f3a-e59a39fdee90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:58.965176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa9778-4665-454e-9f3a-e59a39fdee90 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:59.019008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 849/1682] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:25:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:25:59.022901 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa9778-4665-454e-9f3a-e59a39fdee90 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:59.030926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 834/1683] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:25:58 2022] POST /identity/v3/users => generated 297 bytes in 712 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:25:59.045583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0c65b-7b5c-4ab1-868e-fb5c2f691d88 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:59.059414 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-28fa9778-4665-454e-9f3a-e59a39fdee90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:25:59.116167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0c65b-7b5c-4ab1-868e-fb5c2f691d88 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:59.141122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5175e6-859e-49c3-a90e-59d2a82daa22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:59.142460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5175e6-859e-49c3-a90e-59d2a82daa22 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:59.142996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5175e6-859e-49c3-a90e-59d2a82daa22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:59.147227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5175e6-859e-49c3-a90e-59d2a82daa22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:59.149171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5175e6-859e-49c3-a90e-59d2a82daa22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:59.159188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5175e6-859e-49c3-a90e-59d2a82daa22 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:59.228226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 835/1684] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:59.247888 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5175e6-859e-49c3-a90e-59d2a82daa22 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:59.310209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5175e6-859e-49c3-a90e-59d2a82daa22 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:59.335725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85f77a-f5bc-4367-b555-c1fcba301e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:59.343374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85f77a-f5bc-4367-b555-c1fcba301e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:59.343928 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85f77a-f5bc-4367-b555-c1fcba301e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:59.345032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef85f77a-f5bc-4367-b555-c1fcba301e30 admin admin] PATH_INFO: `/v3/projects/7d1c59691cc147a69ccb3014f74ec98e/users/62e653ba6dc24816a454b174bf846e10/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:59.413569 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85f77a-f5bc-4367-b555-c1fcba301e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1c59691cc147a69ccb3014f74ec98e, user_id=62e653ba6dc24816a454b174bf846e10, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:59.433516 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef85f77a-f5bc-4367-b555-c1fcba301e30 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:59.446387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:59.491200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 836/1685] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:59 2022] PUT /identity/v3/projects/7d1c59691cc147a69ccb3014f74ec98e/users/62e653ba6dc24816a454b174bf846e10/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:25:59.497425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85f77a-f5bc-4367-b555-c1fcba301e30 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:59.585212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa9778-4665-454e-9f3a-e59a39fdee90 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:59.791729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:25:59.822801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef85f77a-f5bc-4367-b555-c1fcba301e30 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:59.853381 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ba6d1-7059-4ce2-93c0-760f8c9ee7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:59.855177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba6d1-7059-4ce2-93c0-760f8c9ee7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:59.855669 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba6d1-7059-4ce2-93c0-760f8c9ee7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:59.857721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ba6d1-7059-4ce2-93c0-760f8c9ee7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:59.861831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ba6d1-7059-4ce2-93c0-760f8c9ee7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:25:59.868976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ba6d1-7059-4ce2-93c0-760f8c9ee7d5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:25:59.889736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a43566-041c-4ba4-bcfe-881757566ecb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:25:59.893489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a43566-041c-4ba4-bcfe-881757566ecb service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:25:59.894410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a43566-041c-4ba4-bcfe-881757566ecb service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:25:59.895098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a43566-041c-4ba4-bcfe-881757566ecb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:25:59.944238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 837/1686] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:25:59.958179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ba6d1-7059-4ce2-93c0-760f8c9ee7d5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:25:59.986494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ba6d1-7059-4ce2-93c0-760f8c9ee7d5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:25:59.998799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb1465-0817-41e7-99af-758af38b40b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:00.003094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1465-0817-41e7-99af-758af38b40b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:00.012230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1465-0817-41e7-99af-758af38b40b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:00.013264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb1465-0817-41e7-99af-758af38b40b4 admin admin] PATH_INFO: `/v3/projects/7d1c59691cc147a69ccb3014f74ec98e/users/62e653ba6dc24816a454b174bf846e10/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:00.029431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb1465-0817-41e7-99af-758af38b40b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1c59691cc147a69ccb3014f74ec98e, user_id=62e653ba6dc24816a454b174bf846e10, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:00.054819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb1465-0817-41e7-99af-758af38b40b4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:00.097399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 838/1687] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:25:59 2022] PUT /identity/v3/projects/7d1c59691cc147a69ccb3014f74ec98e/users/62e653ba6dc24816a454b174bf846e10/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:00.119629 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed093342-a391-47e6-b29c-50696d177289 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:00.120128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed093342-a391-47e6-b29c-50696d177289 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:00.120621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed093342-a391-47e6-b29c-50696d177289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:00.188652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ed093342-a391-47e6-b29c-50696d177289 None None] MFA Rules not processed for user `62e653ba6dc24816a454b174bf846e10`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:00.190639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ed093342-a391-47e6-b29c-50696d177289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:00.225303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a43566-041c-4ba4-bcfe-881757566ecb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:00.227241 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a43566-041c-4ba4-bcfe-881757566ecb service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:00.239099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 850/1688] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:25:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:26:00.713033 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 839/1689] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 595 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:26:00.724574 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a43566-041c-4ba4-bcfe-881757566ecb service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:00.726944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-59a43566-041c-4ba4-bcfe-881757566ecb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:00.871971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a43566-041c-4ba4-bcfe-881757566ecb service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:00.890150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed093342-a391-47e6-b29c-50696d177289 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:00.893074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ed093342-a391-47e6-b29c-50696d177289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:01.126519 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c5855-d9d1-43b7-b3f1-e69c7d951181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:01.128075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c5855-d9d1-43b7-b3f1-e69c7d951181 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:01.128926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c5855-d9d1-43b7-b3f1-e69c7d951181 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:01.129544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c5855-d9d1-43b7-b3f1-e69c7d951181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:01.164191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed093342-a391-47e6-b29c-50696d177289 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:01.185790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:01.186711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c5855-d9d1-43b7-b3f1-e69c7d951181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:01.190660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:01.190660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:01.190660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] PATH_INFO: `/v3/projects/5b8be62fec184cf2872763f6e73b520d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:01.192584 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8be62fec184cf2872763f6e73b520d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:01.194727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c5855-d9d1-43b7-b3f1-e69c7d951181 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:01.197774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:01.218283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:01.268537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 851/1690] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:00 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:01.273285 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1cdb3-b8ed-496b-a7d7-07cb48821bca None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:01.274074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1cdb3-b8ed-496b-a7d7-07cb48821bca None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:01.274512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b1cdb3-b8ed-496b-a7d7-07cb48821bca None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:01.279621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 852/1691] 10.222.0.64 () {54 vars in 912 bytes} [Wed Jul 20 08:26:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:26:01.289726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb219655-abbd-4420-bd20-d895fe672708 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:01.291161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb219655-abbd-4420-bd20-d895fe672708 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:01.293305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb219655-abbd-4420-bd20-d895fe672708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:01.374974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-eb219655-abbd-4420-bd20-d895fe672708 None None] MFA Rules not processed for user `a3b03383554346138b98becb2828851a`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:01.377699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eb219655-abbd-4420-bd20-d895fe672708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:01.474529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8be62fec184cf2872763f6e73b520d', 'request_id': 'req-3495a21f-5eca-4112-9dc9-a27576fe68eb'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:01.496680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:01.512737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8be62fec184cf2872763f6e73b520d', 'request_id': 'req-3495a21f-5eca-4112-9dc9-a27576fe68eb'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:01.541601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 853/1692] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:26:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:01.577387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8be62fec184cf2872763f6e73b520d', 'request_id': 'req-3495a21f-5eca-4112-9dc9-a27576fe68eb'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:01.629368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] The token cache is being invalidate because project 5b8be62fec184cf2872763f6e73b520d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:01.631576 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:01.635505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:01.640556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 840/1693] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:00 2022] DELETE /identity/v3/projects/5b8be62fec184cf2872763f6e73b520d => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:26:01.656347 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d331d9-d308-4db3-964f-e480599133c1 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:01.658765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d331d9-d308-4db3-964f-e480599133c1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:01.659499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d331d9-d308-4db3-964f-e480599133c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:01.664881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:01.675484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:01.721935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-16d331d9-d308-4db3-964f-e480599133c1 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:01.726472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-16d331d9-d308-4db3-964f-e480599133c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:01.727057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3495a21f-5eca-4112-9dc9-a27576fe68eb admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:01.739482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9c233-11d8-41c8-81a1-458921274e0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:01.740902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9c233-11d8-41c8-81a1-458921274e0f service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:01.741364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9c233-11d8-41c8-81a1-458921274e0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:01.741901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9c233-11d8-41c8-81a1-458921274e0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:01.749232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-41a9c233-11d8-41c8-81a1-458921274e0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:01.786161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9c233-11d8-41c8-81a1-458921274e0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:01.789809 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9c233-11d8-41c8-81a1-458921274e0f service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:01.847731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 841/1694] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:01.849239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9c233-11d8-41c8-81a1-458921274e0f service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:01.857323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-41a9c233-11d8-41c8-81a1-458921274e0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:01.954710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 854/1695] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:26:01.957049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4138dfa-4d5d-420d-b8fa-c5c5e6b41c1f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:01.957645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4138dfa-4d5d-420d-b8fa-c5c5e6b41c1f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:01.958201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4138dfa-4d5d-420d-b8fa-c5c5e6b41c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:02.032412 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d4138dfa-4d5d-420d-b8fa-c5c5e6b41c1f None None] MFA Rules not processed for user `a3b03383554346138b98becb2828851a`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:02.043294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d4138dfa-4d5d-420d-b8fa-c5c5e6b41c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:02.078276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 855/1696] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Jul 20 08:26:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:02.080567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd03b32-15db-4bea-811c-13fc5aeb6598 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:02.081111 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd03b32-15db-4bea-811c-13fc5aeb6598 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:02.081555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd03b32-15db-4bea-811c-13fc5aeb6598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:02.117260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9c233-11d8-41c8-81a1-458921274e0f service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:02.129336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66b818-6287-43c9-9559-264b336676bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:02.134938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66b818-6287-43c9-9559-264b336676bf service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:02.135641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66b818-6287-43c9-9559-264b336676bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:02.136400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad66b818-6287-43c9-9559-264b336676bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:02.141596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9fd03b32-15db-4bea-811c-13fc5aeb6598 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:02.144149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9fd03b32-15db-4bea-811c-13fc5aeb6598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:02.145714 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ad66b818-6287-43c9-9559-264b336676bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:02.224795 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 856/1697] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:02.226640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd03b32-15db-4bea-811c-13fc5aeb6598 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:02.233373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9fd03b32-15db-4bea-811c-13fc5aeb6598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:02.321027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd03b32-15db-4bea-811c-13fc5aeb6598 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:02.330899 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced11597-5fb9-4324-a74e-0409438f346d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:02.331942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced11597-5fb9-4324-a74e-0409438f346d service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:02.332504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced11597-5fb9-4324-a74e-0409438f346d service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:02.334390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced11597-5fb9-4324-a74e-0409438f346d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:02.354920 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced11597-5fb9-4324-a74e-0409438f346d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:02.358992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced11597-5fb9-4324-a74e-0409438f346d service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:02.384016 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 857/1698] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:02.387188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced11597-5fb9-4324-a74e-0409438f346d service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:02.423623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced11597-5fb9-4324-a74e-0409438f346d service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:02.438869 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0de69-a296-46b5-a1d4-ebab0c994a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:02.440959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0de69-a296-46b5-a1d4-ebab0c994a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:02.441524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66b818-6287-43c9-9559-264b336676bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:02.441832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0de69-a296-46b5-a1d4-ebab0c994a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:02.442448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e0de69-a296-46b5-a1d4-ebab0c994a3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:02.444406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0de69-a296-46b5-a1d4-ebab0c994a3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:26:02.445067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad66b818-6287-43c9-9559-264b336676bf service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:02.445552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0de69-a296-46b5-a1d4-ebab0c994a3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:02.451420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e0de69-a296-46b5-a1d4-ebab0c994a3a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:02.490676 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 858/1699] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:26:02.494492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 842/1700] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:02.500406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0de69-a296-46b5-a1d4-ebab0c994a3a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:02.558135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e0de69-a296-46b5-a1d4-ebab0c994a3a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:02.585232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea7a46-e92c-496d-af77-1cfe47ea653b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:02.587214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea7a46-e92c-496d-af77-1cfe47ea653b admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:02.587732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea7a46-e92c-496d-af77-1cfe47ea653b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:02.588268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea7a46-e92c-496d-af77-1cfe47ea653b admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:02.589627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea7a46-e92c-496d-af77-1cfe47ea653b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:02.608290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea7a46-e92c-496d-af77-1cfe47ea653b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:02.699904 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 859/1701] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:26:02 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:02.701025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66b818-6287-43c9-9559-264b336676bf service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:02.731410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad66b818-6287-43c9-9559-264b336676bf service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:02.845543 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:02.917765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060962e8-59cc-4b8f-9292-6c2d803695b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:02.921988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060962e8-59cc-4b8f-9292-6c2d803695b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:02.921988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060962e8-59cc-4b8f-9292-6c2d803695b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:02.921988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060962e8-59cc-4b8f-9292-6c2d803695b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:02.921988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060962e8-59cc-4b8f-9292-6c2d803695b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:02.921988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060962e8-59cc-4b8f-9292-6c2d803695b1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:02.972689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 843/1702] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:26:02 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:02.978572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea7a46-e92c-496d-af77-1cfe47ea653b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:03.011015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea7a46-e92c-496d-af77-1cfe47ea653b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:03.032501 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c027cce-f28f-44b6-9454-79f3c4c2c17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:03.039398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c027cce-f28f-44b6-9454-79f3c4c2c17b admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:03.040745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c027cce-f28f-44b6-9454-79f3c4c2c17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:03.042019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c027cce-f28f-44b6-9454-79f3c4c2c17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:03.046252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c027cce-f28f-44b6-9454-79f3c4c2c17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:03.053179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c027cce-f28f-44b6-9454-79f3c4c2c17b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:03.101680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 860/1703] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:03.106002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060962e8-59cc-4b8f-9292-6c2d803695b1 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:03.120546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060962e8-59cc-4b8f-9292-6c2d803695b1 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:03.162956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57093cef-e86e-4870-80ed-2c6261a95ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:03.165137 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57093cef-e86e-4870-80ed-2c6261a95ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:03.167171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57093cef-e86e-4870-80ed-2c6261a95ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:03.169162 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57093cef-e86e-4870-80ed-2c6261a95ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:03.182515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57093cef-e86e-4870-80ed-2c6261a95ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:03.192357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57093cef-e86e-4870-80ed-2c6261a95ba1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:03.245740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 844/1704] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:03.253312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c027cce-f28f-44b6-9454-79f3c4c2c17b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:03.312114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c027cce-f28f-44b6-9454-79f3c4c2c17b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:03.334724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f2685-6e31-4794-9c24-1f5ce5310e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:03.335757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f2685-6e31-4794-9c24-1f5ce5310e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:03.336125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f2685-6e31-4794-9c24-1f5ce5310e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:03.336495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f2685-6e31-4794-9c24-1f5ce5310e5b admin admin] PATH_INFO: `/v3/projects/4076eeb49cca4d79ad2d95b95df2be92/users/8a005cce3277429d8874763c6f15921f/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:03.404417 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f2685-6e31-4794-9c24-1f5ce5310e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4076eeb49cca4d79ad2d95b95df2be92, user_id=8a005cce3277429d8874763c6f15921f, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:03.409083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f2685-6e31-4794-9c24-1f5ce5310e5b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:03.450376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 861/1705] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:03 2022] PUT /identity/v3/projects/4076eeb49cca4d79ad2d95b95df2be92/users/8a005cce3277429d8874763c6f15921f/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:03.461486 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487232bd-7476-4843-94df-2d20679bbf36 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:03.461763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487232bd-7476-4843-94df-2d20679bbf36 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:03.462176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487232bd-7476-4843-94df-2d20679bbf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:03.622572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-487232bd-7476-4843-94df-2d20679bbf36 None None] MFA Rules not processed for user `8a005cce3277429d8874763c6f15921f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:03.639052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-487232bd-7476-4843-94df-2d20679bbf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:03.940114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 845/1706] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:03.946525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f2685-6e31-4794-9c24-1f5ce5310e5b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:04.236745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:04.303455 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f2685-6e31-4794-9c24-1f5ce5310e5b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:04.356176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3316-31d5-4fdb-99c6-fd00bb3c3afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:04.357358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3316-31d5-4fdb-99c6-fd00bb3c3afa admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:04.387517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3316-31d5-4fdb-99c6-fd00bb3c3afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:04.389011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a3316-31d5-4fdb-99c6-fd00bb3c3afa admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:04.397183 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a3316-31d5-4fdb-99c6-fd00bb3c3afa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:04.413869 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656a3316-31d5-4fdb-99c6-fd00bb3c3afa admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:04.681721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487232bd-7476-4843-94df-2d20679bbf36 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:04.708570 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 862/1707] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:26:03 2022] POST /identity/v3/projects => generated 385 bytes in 762 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:26:04.725851 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487232bd-7476-4843-94df-2d20679bbf36 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:05.102964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fadf9-4e86-477f-bb4d-e1fbc6184ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:05.112106 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fadf9-4e86-477f-bb4d-e1fbc6184ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:05.115285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fadf9-4e86-477f-bb4d-e1fbc6184ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:05.119744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14fadf9-4e86-477f-bb4d-e1fbc6184ae8 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:05.126694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fadf9-4e86-477f-bb4d-e1fbc6184ae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:05.155609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14fadf9-4e86-477f-bb4d-e1fbc6184ae8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:05.304545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 846/1708] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:26:04 2022] POST /identity/v3/users => generated 299 bytes in 623 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:26:05.311885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3316-31d5-4fdb-99c6-fd00bb3c3afa admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:05.383505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a3316-31d5-4fdb-99c6-fd00bb3c3afa admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:05.449060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cfd93-9805-4825-8865-cfc9698d7cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:05.450652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cfd93-9805-4825-8865-cfc9698d7cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:05.451452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cfd93-9805-4825-8865-cfc9698d7cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:05.454696 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017cfd93-9805-4825-8865-cfc9698d7cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:05.457026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cfd93-9805-4825-8865-cfc9698d7cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:05.460937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017cfd93-9805-4825-8865-cfc9698d7cf3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:05.523490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 863/1709] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:26:05.533209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fadf9-4e86-477f-bb4d-e1fbc6184ae8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:05.609268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14fadf9-4e86-477f-bb4d-e1fbc6184ae8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:05.662224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46214fe8-f5f7-4a2b-8e78-0ace5bc563f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:05.675255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46214fe8-f5f7-4a2b-8e78-0ace5bc563f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:05.677081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46214fe8-f5f7-4a2b-8e78-0ace5bc563f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:05.677733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46214fe8-f5f7-4a2b-8e78-0ace5bc563f5 admin admin] PATH_INFO: `/v3/projects/7b4c05ac14404ddf8fb88850250d03e4/users/0963ffd9d2d543e0b72fb18446b8c3d5/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:05.804369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46214fe8-f5f7-4a2b-8e78-0ace5bc563f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4c05ac14404ddf8fb88850250d03e4, user_id=0963ffd9d2d543e0b72fb18446b8c3d5, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:05.826633 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46214fe8-f5f7-4a2b-8e78-0ace5bc563f5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:05.878781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 847/1710] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:05 2022] PUT /identity/v3/projects/7b4c05ac14404ddf8fb88850250d03e4/users/0963ffd9d2d543e0b72fb18446b8c3d5/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:05.881466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cfd93-9805-4825-8865-cfc9698d7cf3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:05.958526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017cfd93-9805-4825-8865-cfc9698d7cf3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:06.242522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b9272-b4e4-4ad4-a968-a866eed176e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:06.243229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b9272-b4e4-4ad4-a968-a866eed176e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:06.243581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b9272-b4e4-4ad4-a968-a866eed176e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:06.243874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b9272-b4e4-4ad4-a968-a866eed176e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:06.244692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b9272-b4e4-4ad4-a968-a866eed176e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:06.247554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b9272-b4e4-4ad4-a968-a866eed176e0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:06.263549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 864/1711] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:06.271100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46214fe8-f5f7-4a2b-8e78-0ace5bc563f5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:06.296316 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46214fe8-f5f7-4a2b-8e78-0ace5bc563f5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:06.321639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559812d-24ca-4172-8dfd-cd38f417436f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:06.327776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559812d-24ca-4172-8dfd-cd38f417436f admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:06.329681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559812d-24ca-4172-8dfd-cd38f417436f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:06.330318 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559812d-24ca-4172-8dfd-cd38f417436f admin admin] PATH_INFO: `/v3/projects/7b4c05ac14404ddf8fb88850250d03e4/users/0963ffd9d2d543e0b72fb18446b8c3d5/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:06.340262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559812d-24ca-4172-8dfd-cd38f417436f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b4c05ac14404ddf8fb88850250d03e4, user_id=0963ffd9d2d543e0b72fb18446b8c3d5, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:06.378409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b559812d-24ca-4172-8dfd-cd38f417436f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:06.395414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:06.463485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 848/1712] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:06 2022] PUT /identity/v3/projects/7b4c05ac14404ddf8fb88850250d03e4/users/0963ffd9d2d543e0b72fb18446b8c3d5/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:06.474300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee058d54-39a3-4bf8-be5e-2cda5c3562cf None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:06.477730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee058d54-39a3-4bf8-be5e-2cda5c3562cf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:06.477730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee058d54-39a3-4bf8-be5e-2cda5c3562cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:06.689444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ee058d54-39a3-4bf8-be5e-2cda5c3562cf None None] MFA Rules not processed for user `0963ffd9d2d543e0b72fb18446b8c3d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:06.701146 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ee058d54-39a3-4bf8-be5e-2cda5c3562cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:07.213320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 865/1713] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 740 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:07.264562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559812d-24ca-4172-8dfd-cd38f417436f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:07.504721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b559812d-24ca-4172-8dfd-cd38f417436f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:07.543434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59929afe-186a-4ef6-b63a-8a358314720f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:07.544704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59929afe-186a-4ef6-b63a-8a358314720f service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:07.545246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59929afe-186a-4ef6-b63a-8a358314720f service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:07.545711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59929afe-186a-4ef6-b63a-8a358314720f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:07.612926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59929afe-186a-4ef6-b63a-8a358314720f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:07.624362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59929afe-186a-4ef6-b63a-8a358314720f service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:07.684499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 849/1714] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:07.744202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee058d54-39a3-4bf8-be5e-2cda5c3562cf None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:08.198326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:08.242210 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee058d54-39a3-4bf8-be5e-2cda5c3562cf None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:08.259647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b4e95-e843-42ce-9f01-cba095cb3755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:08.261960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4e95-e843-42ce-9f01-cba095cb3755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:08.264994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4e95-e843-42ce-9f01-cba095cb3755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:08.265342 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4e95-e843-42ce-9f01-cba095cb3755 admin admin] PATH_INFO: `/v3/users/8a005cce3277429d8874763c6f15921f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:08.269131 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b4e95-e843-42ce-9f01-cba095cb3755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a005cce3277429d8874763c6f15921f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:08.276851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b4e95-e843-42ce-9f01-cba095cb3755 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:08.513386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-126b4e95-e843-42ce-9f01-cba095cb3755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a005cce3277429d8874763c6f15921f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:08.563183 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-126b4e95-e843-42ce-9f01-cba095cb3755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a005cce3277429d8874763c6f15921f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:08.564483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-126b4e95-e843-42ce-9f01-cba095cb3755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a005cce3277429d8874763c6f15921f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:08.581836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-126b4e95-e843-42ce-9f01-cba095cb3755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a005cce3277429d8874763c6f15921f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:08.622734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-126b4e95-e843-42ce-9f01-cba095cb3755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a005cce3277429d8874763c6f15921f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:08.658892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 866/1715] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:07 2022] DELETE /identity/v3/users/8a005cce3277429d8874763c6f15921f => generated 0 bytes in 916 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:08.666429 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf20edb-20ee-4781-9cd2-23ef8f75b296 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:08.668245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf20edb-20ee-4781-9cd2-23ef8f75b296 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:08.668991 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf20edb-20ee-4781-9cd2-23ef8f75b296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:08.773574 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-baf20edb-20ee-4781-9cd2-23ef8f75b296 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:08.775922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-baf20edb-20ee-4781-9cd2-23ef8f75b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:08.944819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 850/1716] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:08.988210 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6add5-3aaa-42c6-9780-5358a1b9a5c5 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:08.988923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6add5-3aaa-42c6-9780-5358a1b9a5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:08.989063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e6add5-3aaa-42c6-9780-5358a1b9a5c5 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:08.990980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 867/1717] 10.222.0.64 () {54 vars in 921 bytes} [Wed Jul 20 08:26:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:26:09.016520 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e78f7e-9f25-4f39-8882-9c12c27f5167 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:09.017932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6add5-3aaa-42c6-9780-5358a1b9a5c5 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:09.027607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e78f7e-9f25-4f39-8882-9c12c27f5167 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:09.028579 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e78f7e-9f25-4f39-8882-9c12c27f5167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:09.029639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-33e6add5-3aaa-42c6-9780-5358a1b9a5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:09.113318 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-63e78f7e-9f25-4f39-8882-9c12c27f5167 None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:09.118141 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-63e78f7e-9f25-4f39-8882-9c12c27f5167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:09.278297 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e6add5-3aaa-42c6-9780-5358a1b9a5c5 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:09.283820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6963779-9905-4416-8d61-77dad634cce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:09.285029 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6963779-9905-4416-8d61-77dad634cce2 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:09.285330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6963779-9905-4416-8d61-77dad634cce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:09.285609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6963779-9905-4416-8d61-77dad634cce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:09.303151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6963779-9905-4416-8d61-77dad634cce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:09.305629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6963779-9905-4416-8d61-77dad634cce2 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:09.330465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 868/1718] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:09.424506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 851/1719] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Jul 20 08:26:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:26:09.451490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6963779-9905-4416-8d61-77dad634cce2 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:09.453217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f6963779-9905-4416-8d61-77dad634cce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:09.507923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6963779-9905-4416-8d61-77dad634cce2 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:09.543373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddc28f-df61-4b84-9824-b97dbf748614 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:09.548554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddc28f-df61-4b84-9824-b97dbf748614 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:09.549175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddc28f-df61-4b84-9824-b97dbf748614 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:09.549685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddc28f-df61-4b84-9824-b97dbf748614 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:09.627806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dddc28f-df61-4b84-9824-b97dbf748614 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:09.631155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dddc28f-df61-4b84-9824-b97dbf748614 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:09.689309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 869/1720] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:09.967001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bcc3b-8bc0-4f44-b114-a532a46e10de None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:09.967572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bcc3b-8bc0-4f44-b114-a532a46e10de None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:09.967937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bcc3b-8bc0-4f44-b114-a532a46e10de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:10.072789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2b0bcc3b-8bc0-4f44-b114-a532a46e10de None None] MFA Rules not processed for user `e6c2070c6d2a450c887b8ca6b97c4b38`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:10.075435 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2b0bcc3b-8bc0-4f44-b114-a532a46e10de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:10.160607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:10.391176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddc28f-df61-4b84-9824-b97dbf748614 service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:10.394477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8dddc28f-df61-4b84-9824-b97dbf748614 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:10.474159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dddc28f-df61-4b84-9824-b97dbf748614 service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:10.508609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:10.510259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:10.521367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:10.522099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] PATH_INFO: `/v3/projects/4076eeb49cca4d79ad2d95b95df2be92` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:10.549430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4076eeb49cca4d79ad2d95b95df2be92)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:10.551992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: WARNING py.warnings [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 08:26:10.551992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: warnings.warn(msg) Jul 20 08:26:10.551992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: Jul 20 08:26:10.558869 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:10.709132 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 852/1721] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 743 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:10.766851 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bcc3b-8bc0-4f44-b114-a532a46e10de None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:10.770894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2b0bcc3b-8bc0-4f44-b114-a532a46e10de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:10.860144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4076eeb49cca4d79ad2d95b95df2be92', 'request_id': 'req-5743c6b5-b94e-437c-bf63-078917ef8c99'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:10.865261 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:10.884733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4076eeb49cca4d79ad2d95b95df2be92', 'request_id': 'req-5743c6b5-b94e-437c-bf63-078917ef8c99'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:10.893126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0bcc3b-8bc0-4f44-b114-a532a46e10de None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:10.966172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4076eeb49cca4d79ad2d95b95df2be92', 'request_id': 'req-5743c6b5-b94e-437c-bf63-078917ef8c99'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:11.059067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] The token cache is being invalidate because project 4076eeb49cca4d79ad2d95b95df2be92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:11.059555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:11.069467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:11.091656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 870/1722] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:10 2022] DELETE /identity/v3/projects/4076eeb49cca4d79ad2d95b95df2be92 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:11.093338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:11.106939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:11.168626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9109a4ed-c237-452b-9f1a-017cd20d4e1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:11.173433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109a4ed-c237-452b-9f1a-017cd20d4e1f service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:11.180621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109a4ed-c237-452b-9f1a-017cd20d4e1f service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:11.181355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109a4ed-c237-452b-9f1a-017cd20d4e1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:11.191686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9109a4ed-c237-452b-9f1a-017cd20d4e1f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:11.435627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109a4ed-c237-452b-9f1a-017cd20d4e1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:11.438294 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9109a4ed-c237-452b-9f1a-017cd20d4e1f service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:11.467570 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 853/1723] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:26:11.484725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743c6b5-b94e-437c-bf63-078917ef8c99 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:11.492572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:11.493604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:11.493959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:11.494316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] PATH_INFO: `/v3/users/0963ffd9d2d543e0b72fb18446b8c3d5` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:11.495802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0963ffd9d2d543e0b72fb18446b8c3d5)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:11.497962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:11.651946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0963ffd9d2d543e0b72fb18446b8c3d5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:11.779762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0963ffd9d2d543e0b72fb18446b8c3d5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:11.787411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0963ffd9d2d543e0b72fb18446b8c3d5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:11.810487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0963ffd9d2d543e0b72fb18446b8c3d5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:11.827140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0963ffd9d2d543e0b72fb18446b8c3d5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:11.839549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 871/1724] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:11 2022] DELETE /identity/v3/users/0963ffd9d2d543e0b72fb18446b8c3d5 => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:15.670024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf371b5d-4ad2-4482-9782-86f3f013b262 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:15.671044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf371b5d-4ad2-4482-9782-86f3f013b262 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:15.671044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf371b5d-4ad2-4482-9782-86f3f013b262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:15.745521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-bf371b5d-4ad2-4482-9782-86f3f013b262 None None] MFA Rules not processed for user `5bf1a059ddb34c2d8181712f53d592c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:15.747206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bf371b5d-4ad2-4482-9782-86f3f013b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:15.836799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:15.839339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:15.902359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:15.976895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 854/1725] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:15.990197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a94bc-ee42-46aa-9b02-536f9427bdce None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:15.991220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a94bc-ee42-46aa-9b02-536f9427bdce None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:15.991575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a94bc-ee42-46aa-9b02-536f9427bdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:16.030965 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5a5de-9a05-4cec-8a7d-36a971797b88 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:16.037794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:16.039428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:16.040006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:16.040461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] PATH_INFO: `/v3/projects/7b4c05ac14404ddf8fb88850250d03e4` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:16.042036 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b4c05ac14404ddf8fb88850250d03e4)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:16.044564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:16.048041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:16.100210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c21a94bc-ee42-46aa-9b02-536f9427bdce None None] MFA Rules not processed for user `5bf1a059ddb34c2d8181712f53d592c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:16.107919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c21a94bc-ee42-46aa-9b02-536f9427bdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:16.118061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b4c05ac14404ddf8fb88850250d03e4', 'request_id': 'req-c51c8b74-823a-42fc-acf5-74ba7849c228'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:16.132420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b4c05ac14404ddf8fb88850250d03e4', 'request_id': 'req-c51c8b74-823a-42fc-acf5-74ba7849c228'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:16.143217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b4c05ac14404ddf8fb88850250d03e4', 'request_id': 'req-c51c8b74-823a-42fc-acf5-74ba7849c228'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:16.159485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] The token cache is being invalidate because project 7b4c05ac14404ddf8fb88850250d03e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:16.160143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:16.161088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c51c8b74-823a-42fc-acf5-74ba7849c228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:16.163484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 872/1726] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:15 2022] DELETE /identity/v3/projects/7b4c05ac14404ddf8fb88850250d03e4 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:16.174665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd9e31-f282-4163-a9c3-c6d62d415b95 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:16.175356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd9e31-f282-4163-a9c3-c6d62d415b95 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:16.177089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd9e31-f282-4163-a9c3-c6d62d415b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:16.215012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-eebd9e31-f282-4163-a9c3-c6d62d415b95 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:16.217648 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eebd9e31-f282-4163-a9c3-c6d62d415b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:16.386130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 855/1727] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:16.410335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a94bc-ee42-46aa-9b02-536f9427bdce None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:16.413363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c21a94bc-ee42-46aa-9b02-536f9427bdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:16.418319 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 873/1728] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:16.429308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39889e-7595-4045-bc21-f5565411f564 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:16.429768 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39889e-7595-4045-bc21-f5565411f564 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:16.430163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39889e-7595-4045-bc21-f5565411f564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:16.466812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4a39889e-7595-4045-bc21-f5565411f564 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:16.468398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4a39889e-7595-4045-bc21-f5565411f564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:16.488894 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 874/1729] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:26:16.493037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39889e-7595-4045-bc21-f5565411f564 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:16.520869 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39889e-7595-4045-bc21-f5565411f564 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:16.533726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858ff6d-df74-4c56-9040-c9e8fa7ee6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:16.537177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858ff6d-df74-4c56-9040-c9e8fa7ee6be admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:16.538144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858ff6d-df74-4c56-9040-c9e8fa7ee6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:16.538918 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858ff6d-df74-4c56-9040-c9e8fa7ee6be admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:16.541279 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858ff6d-df74-4c56-9040-c9e8fa7ee6be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:26:16.541859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858ff6d-df74-4c56-9040-c9e8fa7ee6be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:16.548467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8858ff6d-df74-4c56-9040-c9e8fa7ee6be admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:16.592374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 875/1730] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:26:16.596751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c2b20-ad90-4e70-bcb6-4a55e885e8da None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:16.597316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c2b20-ad90-4e70-bcb6-4a55e885e8da None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:16.598483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c2b20-ad90-4e70-bcb6-4a55e885e8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:16.644243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a94bc-ee42-46aa-9b02-536f9427bdce None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:16.653991 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ab27a-ba4b-4ad1-a2fb-c4ad5fa3d617 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:16.655116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ab27a-ba4b-4ad1-a2fb-c4ad5fa3d617 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:16.655641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ab27a-ba4b-4ad1-a2fb-c4ad5fa3d617 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:16.656177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ab27a-ba4b-4ad1-a2fb-c4ad5fa3d617 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:16.659797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-633ab27a-ba4b-4ad1-a2fb-c4ad5fa3d617 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:16.690277 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ab27a-ba4b-4ad1-a2fb-c4ad5fa3d617 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:16.695044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ab27a-ba4b-4ad1-a2fb-c4ad5fa3d617 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:16.701539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a83c2b20-ad90-4e70-bcb6-4a55e885e8da None None] MFA Rules not processed for user `b2326ff15f294e15806d8a564770b35e`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:16.705117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a83c2b20-ad90-4e70-bcb6-4a55e885e8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:16.711939 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 856/1731] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:26:16.713007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ab27a-ba4b-4ad1-a2fb-c4ad5fa3d617 service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:16.730929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ab27a-ba4b-4ad1-a2fb-c4ad5fa3d617 service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:16.749289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302c713-f98f-4bf5-aa57-8574ce054836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:16.749993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302c713-f98f-4bf5-aa57-8574ce054836 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:16.752758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302c713-f98f-4bf5-aa57-8574ce054836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:16.754772 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302c713-f98f-4bf5-aa57-8574ce054836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:16.755632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302c713-f98f-4bf5-aa57-8574ce054836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:16.760213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302c713-f98f-4bf5-aa57-8574ce054836 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:16.815610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 857/1732] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:26:16 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:16.819922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302c713-f98f-4bf5-aa57-8574ce054836 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:16.925019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:16.932670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302c713-f98f-4bf5-aa57-8574ce054836 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:16.945501 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a027d60-29ad-40d3-bffd-75935be54424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:16.946468 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a027d60-29ad-40d3-bffd-75935be54424 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:16.947284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a027d60-29ad-40d3-bffd-75935be54424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:16.947916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a027d60-29ad-40d3-bffd-75935be54424 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:16.948819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a027d60-29ad-40d3-bffd-75935be54424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:16.951855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a027d60-29ad-40d3-bffd-75935be54424 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:16.959009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 876/1733] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:16.982933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 858/1734] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:26:16 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:16.988913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c2b20-ad90-4e70-bcb6-4a55e885e8da None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:17.015559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:17.052656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c2b20-ad90-4e70-bcb6-4a55e885e8da None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:17.072310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01da38a-2715-439a-b9d1-af7e2a945583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:17.077046 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01da38a-2715-439a-b9d1-af7e2a945583 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:17.077763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01da38a-2715-439a-b9d1-af7e2a945583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:17.079252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01da38a-2715-439a-b9d1-af7e2a945583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:17.081837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01da38a-2715-439a-b9d1-af7e2a945583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:17.087882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01da38a-2715-439a-b9d1-af7e2a945583 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:17.121020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 877/1735] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:17.125613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a027d60-29ad-40d3-bffd-75935be54424 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:17.149336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a027d60-29ad-40d3-bffd-75935be54424 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:17.165930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f721b4a-9597-4a3c-8434-605d049c62c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:17.167012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f721b4a-9597-4a3c-8434-605d049c62c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:17.171509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f721b4a-9597-4a3c-8434-605d049c62c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:17.172929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f721b4a-9597-4a3c-8434-605d049c62c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:17.172929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f721b4a-9597-4a3c-8434-605d049c62c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:17.178636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f721b4a-9597-4a3c-8434-605d049c62c2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:17.202212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 859/1736] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:26:17.216952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01da38a-2715-439a-b9d1-af7e2a945583 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:17.236039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01da38a-2715-439a-b9d1-af7e2a945583 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:17.254650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc60648-4903-47df-a70b-0d4f6871b337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:17.256375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f721b4a-9597-4a3c-8434-605d049c62c2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:17.258406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0f721b4a-9597-4a3c-8434-605d049c62c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:17.273379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc60648-4903-47df-a70b-0d4f6871b337 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:17.273379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc60648-4903-47df-a70b-0d4f6871b337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:17.273379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc60648-4903-47df-a70b-0d4f6871b337 admin admin] PATH_INFO: `/v3/projects/a31d4d7270a942fa84698d04d807d59b/users/b99cc3ebd1674da8936964a837c628f9/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:17.336329 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc60648-4903-47df-a70b-0d4f6871b337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a31d4d7270a942fa84698d04d807d59b, user_id=b99cc3ebd1674da8936964a837c628f9, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:17.340314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc60648-4903-47df-a70b-0d4f6871b337 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:17.460851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 878/1737] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:17 2022] PUT /identity/v3/projects/a31d4d7270a942fa84698d04d807d59b/users/b99cc3ebd1674da8936964a837c628f9/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:17.466528 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618d87a-6f61-4be2-8a5b-ee7a9e3a2e77 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:17.474987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618d87a-6f61-4be2-8a5b-ee7a9e3a2e77 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:17.477436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618d87a-6f61-4be2-8a5b-ee7a9e3a2e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:17.561858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7618d87a-6f61-4be2-8a5b-ee7a9e3a2e77 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:17.565090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7618d87a-6f61-4be2-8a5b-ee7a9e3a2e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:17.659164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f721b4a-9597-4a3c-8434-605d049c62c2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:17.775299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 879/1738] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:17.821175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618d87a-6f61-4be2-8a5b-ee7a9e3a2e77 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:17.823102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7618d87a-6f61-4be2-8a5b-ee7a9e3a2e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:17.924862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6b5ab-9027-421b-963e-247b1ded355e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:17.925585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6b5ab-9027-421b-963e-247b1ded355e service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:17.925731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6b5ab-9027-421b-963e-247b1ded355e service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:17.926060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6b5ab-9027-421b-963e-247b1ded355e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:18.035228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618d87a-6f61-4be2-8a5b-ee7a9e3a2e77 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:18.060688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7266e2b-022a-482f-892a-4774fea32030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:18.062188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7266e2b-022a-482f-892a-4774fea32030 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:18.063453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7266e2b-022a-482f-892a-4774fea32030 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:18.064063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7266e2b-022a-482f-892a-4774fea32030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:18.100469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7266e2b-022a-482f-892a-4774fea32030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:18.103069 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7266e2b-022a-482f-892a-4774fea32030 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:18.118391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6b5ab-9027-421b-963e-247b1ded355e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:18.124252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6b5ab-9027-421b-963e-247b1ded355e service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:18.146030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 880/1739] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:18.150696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 860/1740] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 895 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:26:20.423264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ecbfc-3785-481f-bd58-b97aa2a6553a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:20.424131 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ecbfc-3785-481f-bd58-b97aa2a6553a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:20.424429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ecbfc-3785-481f-bd58-b97aa2a6553a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:20.521375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-510ecbfc-3785-481f-bd58-b97aa2a6553a None None] MFA Rules not processed for user `e6c2070c6d2a450c887b8ca6b97c4b38`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:20.528181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-510ecbfc-3785-481f-bd58-b97aa2a6553a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:20.941147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 881/1741] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:22.551642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f99bcf-6045-4fb0-a05a-ba03acf5cf33 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:22.552327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f99bcf-6045-4fb0-a05a-ba03acf5cf33 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:22.552327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f99bcf-6045-4fb0-a05a-ba03acf5cf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:22.593053 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:22.614494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b0f99bcf-6045-4fb0-a05a-ba03acf5cf33 None None] MFA Rules not processed for user `b99cc3ebd1674da8936964a837c628f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:22.617036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b0f99bcf-6045-4fb0-a05a-ba03acf5cf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:22.783974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 861/1742] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:22.788800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ecbfc-3785-481f-bd58-b97aa2a6553a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:22.820003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ecbfc-3785-481f-bd58-b97aa2a6553a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:22.842557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c06bd-c19b-4306-b793-f6a73b62ad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:22.843278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c06bd-c19b-4306-b793-f6a73b62ad14 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:22.843558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c06bd-c19b-4306-b793-f6a73b62ad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:22.851994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c06bd-c19b-4306-b793-f6a73b62ad14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:22.852749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c06bd-c19b-4306-b793-f6a73b62ad14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:22.870142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c06bd-c19b-4306-b793-f6a73b62ad14 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:22.924857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:22.950723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 882/1743] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:26:22 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:22.954533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f99bcf-6045-4fb0-a05a-ba03acf5cf33 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:22.985288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f99bcf-6045-4fb0-a05a-ba03acf5cf33 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:23.220969 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90087196-60ec-4677-bb38-4039c036384e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:23.226995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90087196-60ec-4677-bb38-4039c036384e admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:23.227481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90087196-60ec-4677-bb38-4039c036384e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:23.227783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90087196-60ec-4677-bb38-4039c036384e admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:23.228595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90087196-60ec-4677-bb38-4039c036384e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:23.235200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90087196-60ec-4677-bb38-4039c036384e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:23.281179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 862/1744] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:26:22 2022] POST /identity/v3/users => generated 292 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:23.286151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c06bd-c19b-4306-b793-f6a73b62ad14 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:23.311820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c06bd-c19b-4306-b793-f6a73b62ad14 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:23.326572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f8ca7-98bd-4096-8115-4f96e25975a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:23.328566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f8ca7-98bd-4096-8115-4f96e25975a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:23.328906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f8ca7-98bd-4096-8115-4f96e25975a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:23.329963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f8ca7-98bd-4096-8115-4f96e25975a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:23.332298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f8ca7-98bd-4096-8115-4f96e25975a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:23.337937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f8ca7-98bd-4096-8115-4f96e25975a9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:23.370946 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 883/1745] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:23.376425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90087196-60ec-4677-bb38-4039c036384e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:23.409912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90087196-60ec-4677-bb38-4039c036384e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:23.426011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb8c41-abca-4a40-84e7-207feb1ba8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:23.427022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb8c41-abca-4a40-84e7-207feb1ba8b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:23.427389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb8c41-abca-4a40-84e7-207feb1ba8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:23.427651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb8c41-abca-4a40-84e7-207feb1ba8b7 admin admin] PATH_INFO: `/v3/projects/ef83da28b788440491781b52fa2f498a/users/7cc1bff8159448db9cc5ee6ab5302ffb/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:23.457018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb8c41-abca-4a40-84e7-207feb1ba8b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef83da28b788440491781b52fa2f498a, user_id=7cc1bff8159448db9cc5ee6ab5302ffb, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:23.469307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb8c41-abca-4a40-84e7-207feb1ba8b7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:23.497627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 863/1746] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:23 2022] PUT /identity/v3/projects/ef83da28b788440491781b52fa2f498a/users/7cc1bff8159448db9cc5ee6ab5302ffb/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:23.504989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f8ca7-98bd-4096-8115-4f96e25975a9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:23.558637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f8ca7-98bd-4096-8115-4f96e25975a9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:23.765094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308155b-e83f-459f-b2c2-629cd8f0c698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:23.766266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308155b-e83f-459f-b2c2-629cd8f0c698 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:23.766714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308155b-e83f-459f-b2c2-629cd8f0c698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:23.767006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308155b-e83f-459f-b2c2-629cd8f0c698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:23.769510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308155b-e83f-459f-b2c2-629cd8f0c698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:23.771615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308155b-e83f-459f-b2c2-629cd8f0c698 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:23.791345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 884/1747] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:23.794089 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb8c41-abca-4a40-84e7-207feb1ba8b7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:23.812130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb8c41-abca-4a40-84e7-207feb1ba8b7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:23.826292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9925d0-8261-4aff-baba-6a419d0b73c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:23.827439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9925d0-8261-4aff-baba-6a419d0b73c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:23.829440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9925d0-8261-4aff-baba-6a419d0b73c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:23.829758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9925d0-8261-4aff-baba-6a419d0b73c5 admin admin] PATH_INFO: `/v3/projects/ef83da28b788440491781b52fa2f498a/users/7cc1bff8159448db9cc5ee6ab5302ffb/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:23.839681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9925d0-8261-4aff-baba-6a419d0b73c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef83da28b788440491781b52fa2f498a, user_id=7cc1bff8159448db9cc5ee6ab5302ffb, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:23.849775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9925d0-8261-4aff-baba-6a419d0b73c5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:23.898524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 864/1748] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:23 2022] PUT /identity/v3/projects/ef83da28b788440491781b52fa2f498a/users/7cc1bff8159448db9cc5ee6ab5302ffb/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:24.672764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c83f-76d3-42d5-88d3-53c58582fd73 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:24.673193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c83f-76d3-42d5-88d3-53c58582fd73 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:24.673394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c83f-76d3-42d5-88d3-53c58582fd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:24.731318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8c34c83f-76d3-42d5-88d3-53c58582fd73 None None] MFA Rules not processed for user `b2326ff15f294e15806d8a564770b35e`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:24.732959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8c34c83f-76d3-42d5-88d3-53c58582fd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:24.915207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 885/1749] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:26.273222 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a46e1b-a77e-480d-a423-99131db3c3bb None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:26.274027 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a46e1b-a77e-480d-a423-99131db3c3bb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:26.275037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a46e1b-a77e-480d-a423-99131db3c3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:26.414431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-89a46e1b-a77e-480d-a423-99131db3c3bb None None] MFA Rules not processed for user `7cc1bff8159448db9cc5ee6ab5302ffb`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:26.416457 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-89a46e1b-a77e-480d-a423-99131db3c3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:26.528328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:26.603172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 865/1750] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:26.621432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34c83f-76d3-42d5-88d3-53c58582fd73 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:26.623841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8c34c83f-76d3-42d5-88d3-53c58582fd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:26.898187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34c83f-76d3-42d5-88d3-53c58582fd73 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:26.933289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33afef-253e-4297-b1b4-c6143c54f897 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:26.939762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33afef-253e-4297-b1b4-c6143c54f897 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:26.940276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33afef-253e-4297-b1b4-c6143c54f897 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:26.941756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33afef-253e-4297-b1b4-c6143c54f897 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:27.126633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:27.270675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33afef-253e-4297-b1b4-c6143c54f897 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:27.283043 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33afef-253e-4297-b1b4-c6143c54f897 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:27.327391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 886/1751] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 710 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:30.487643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ea007-cadf-45f0-988c-461db4c9e7c4 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:30.487643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ea007-cadf-45f0-988c-461db4c9e7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:30.487643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ea007-cadf-45f0-988c-461db4c9e7c4 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:30.490307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 866/1752] 10.222.0.64 () {54 vars in 921 bytes} [Wed Jul 20 08:26:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:26:30.627083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12402e-de15-4849-a9aa-c0476bf4bb8a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:30.627905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12402e-de15-4849-a9aa-c0476bf4bb8a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:30.627905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12402e-de15-4849-a9aa-c0476bf4bb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:30.686936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2b12402e-de15-4849-a9aa-c0476bf4bb8a None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:30.689938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2b12402e-de15-4849-a9aa-c0476bf4bb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:30.864345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 887/1753] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Jul 20 08:26:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:30.880524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ea007-cadf-45f0-988c-461db4c9e7c4 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:30.899806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ea007-cadf-45f0-988c-461db4c9e7c4 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:30.910179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731f0f8-0a9e-4b3e-b416-de3d9c2267c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:30.912631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731f0f8-0a9e-4b3e-b416-de3d9c2267c4 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:30.913914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731f0f8-0a9e-4b3e-b416-de3d9c2267c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:30.914721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731f0f8-0a9e-4b3e-b416-de3d9c2267c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:30.931310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731f0f8-0a9e-4b3e-b416-de3d9c2267c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:30.934856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731f0f8-0a9e-4b3e-b416-de3d9c2267c4 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:30.967186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 867/1754] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:31.058183 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12402e-de15-4849-a9aa-c0476bf4bb8a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:31.061181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2b12402e-de15-4849-a9aa-c0476bf4bb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:31.079545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12402e-de15-4849-a9aa-c0476bf4bb8a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:31.084903 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237166bc-70b3-47e8-8c0c-ce24dc636bf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:31.085751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237166bc-70b3-47e8-8c0c-ce24dc636bf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:31.086059 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237166bc-70b3-47e8-8c0c-ce24dc636bf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:31.086468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237166bc-70b3-47e8-8c0c-ce24dc636bf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:31.101621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237166bc-70b3-47e8-8c0c-ce24dc636bf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:31.104441 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237166bc-70b3-47e8-8c0c-ce24dc636bf2 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:31.122535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 888/1755] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:31 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:36.163579 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731f0f8-0a9e-4b3e-b416-de3d9c2267c4 service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:36.165883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e731f0f8-0a9e-4b3e-b416-de3d9c2267c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:36.279795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731f0f8-0a9e-4b3e-b416-de3d9c2267c4 service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:36.284690 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:36.285606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:36.285897 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:36.286185 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] PATH_INFO: `/v3/users/bb1b6336acb2484e9642db8bf7b21efd` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:36.287493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb1b6336acb2484e9642db8bf7b21efd)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:36.289489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:36.343430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb1b6336acb2484e9642db8bf7b21efd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:36.351246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb1b6336acb2484e9642db8bf7b21efd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:36.361807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb1b6336acb2484e9642db8bf7b21efd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:36.385209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb1b6336acb2484e9642db8bf7b21efd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:36.386481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb1b6336acb2484e9642db8bf7b21efd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:36.398022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 868/1756] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:36 2022] DELETE /identity/v3/users/bb1b6336acb2484e9642db8bf7b21efd => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:36.406501 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751599-8820-43c8-bd72-1cb3ea5bcc40 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:36.407087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751599-8820-43c8-bd72-1cb3ea5bcc40 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:36.407579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49751599-8820-43c8-bd72-1cb3ea5bcc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:36.439251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-49751599-8820-43c8-bd72-1cb3ea5bcc40 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:36.441348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-49751599-8820-43c8-bd72-1cb3ea5bcc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:36.543806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 889/1757] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:36.555452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:36.557206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:36.626808 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:36.649689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92291b5-32ce-4a55-b088-92302cd8431d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:36.655826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971218f-5304-4c51-b0eb-697c94cbd820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:36.658293 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971218f-5304-4c51-b0eb-697c94cbd820 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:36.659161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971218f-5304-4c51-b0eb-697c94cbd820 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:36.659906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971218f-5304-4c51-b0eb-697c94cbd820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:36.680209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971218f-5304-4c51-b0eb-697c94cbd820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:36.684037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971218f-5304-4c51-b0eb-697c94cbd820 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:36.699559 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 869/1758] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:37.388057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49751599-8820-43c8-bd72-1cb3ea5bcc40 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:37.401881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-49751599-8820-43c8-bd72-1cb3ea5bcc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:37.484847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49751599-8820-43c8-bd72-1cb3ea5bcc40 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:37.498357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:37.500753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:37.502124 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:37.503290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] PATH_INFO: `/v3/projects/385a53799ce044f898f742ebd3dd79da` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:37.506066 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] RBAC: Authorizing `identity:delete_project(project_id=385a53799ce044f898f742ebd3dd79da)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:37.509376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:37.660497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '385a53799ce044f898f742ebd3dd79da', 'request_id': 'req-1fa65105-405a-44ed-bed8-d7dd7640d0af'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:37.664382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:37.680323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '385a53799ce044f898f742ebd3dd79da', 'request_id': 'req-1fa65105-405a-44ed-bed8-d7dd7640d0af'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:37.696132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '385a53799ce044f898f742ebd3dd79da', 'request_id': 'req-1fa65105-405a-44ed-bed8-d7dd7640d0af'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:37.737742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] The token cache is being invalidate because project 385a53799ce044f898f742ebd3dd79da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:37.738038 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:37.738957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1fa65105-405a-44ed-bed8-d7dd7640d0af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:37.741627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 890/1759] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:37 2022] DELETE /identity/v3/projects/385a53799ce044f898f742ebd3dd79da => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:37.748323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971218f-5304-4c51-b0eb-697c94cbd820 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:37.767791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971218f-5304-4c51-b0eb-697c94cbd820 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:37.939627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f714c700-bee8-4175-afcc-4a884e9accd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:37.940990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714c700-bee8-4175-afcc-4a884e9accd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:37.941451 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714c700-bee8-4175-afcc-4a884e9accd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:37.941842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f714c700-bee8-4175-afcc-4a884e9accd8 admin admin] PATH_INFO: `/v3/users/62e653ba6dc24816a454b174bf846e10` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:37.945229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714c700-bee8-4175-afcc-4a884e9accd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e653ba6dc24816a454b174bf846e10)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:37.947952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f714c700-bee8-4175-afcc-4a884e9accd8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:38.159494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f714c700-bee8-4175-afcc-4a884e9accd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e653ba6dc24816a454b174bf846e10'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:38.192641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f714c700-bee8-4175-afcc-4a884e9accd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e653ba6dc24816a454b174bf846e10'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:38.221014 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f714c700-bee8-4175-afcc-4a884e9accd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e653ba6dc24816a454b174bf846e10'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:38.268427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f714c700-bee8-4175-afcc-4a884e9accd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e653ba6dc24816a454b174bf846e10'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:38.269807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f714c700-bee8-4175-afcc-4a884e9accd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e653ba6dc24816a454b174bf846e10'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:38.299756 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 870/1760] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:37 2022] DELETE /identity/v3/users/62e653ba6dc24816a454b174bf846e10 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:39.285724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0621f6-4cbb-40b5-9733-b80b2f0d27d2 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:39.286642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0621f6-4cbb-40b5-9733-b80b2f0d27d2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:39.287113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0621f6-4cbb-40b5-9733-b80b2f0d27d2 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:39.288828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 891/1761] 10.222.0.64 () {54 vars in 921 bytes} [Wed Jul 20 08:26:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:26:39.315828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503b7f-70c5-4be8-aa3d-af433826c6e1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:39.316802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503b7f-70c5-4be8-aa3d-af433826c6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:39.317278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d503b7f-70c5-4be8-aa3d-af433826c6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:39.370251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2d503b7f-70c5-4be8-aa3d-af433826c6e1 None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:39.391608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2d503b7f-70c5-4be8-aa3d-af433826c6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:39.540181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:39.564595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 871/1762] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Jul 20 08:26:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:39.863737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0621f6-4cbb-40b5-9733-b80b2f0d27d2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:39.865499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9e0621f6-4cbb-40b5-9733-b80b2f0d27d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:39.896538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0621f6-4cbb-40b5-9733-b80b2f0d27d2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:39.906026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c8831-f3b4-4549-aed8-99e5c31990bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:39.908658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c8831-f3b4-4549-aed8-99e5c31990bb service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:39.909142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c8831-f3b4-4549-aed8-99e5c31990bb service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:39.909746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c8831-f3b4-4549-aed8-99e5c31990bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:39.927963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c8831-f3b4-4549-aed8-99e5c31990bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:39.930345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c8831-f3b4-4549-aed8-99e5c31990bb service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:39.951694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 892/1763] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:40.234185 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d503b7f-70c5-4be8-aa3d-af433826c6e1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:40.236681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2d503b7f-70c5-4be8-aa3d-af433826c6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:40.330729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c8831-f3b4-4549-aed8-99e5c31990bb service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:40.332954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ea0c8831-f3b4-4549-aed8-99e5c31990bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:40.471893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d503b7f-70c5-4be8-aa3d-af433826c6e1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:40.486977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c8831-f3b4-4549-aed8-99e5c31990bb service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:40.495800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:40.499667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:40.500387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:40.501233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] PATH_INFO: `/v3/projects/7d1c59691cc147a69ccb3014f74ec98e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:40.505078 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d1c59691cc147a69ccb3014f74ec98e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:40.509556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef2b1f-60d7-4774-9fa4-fa72ba95629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:40.510478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef2b1f-60d7-4774-9fa4-fa72ba95629e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:40.510898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef2b1f-60d7-4774-9fa4-fa72ba95629e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:40.511189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef2b1f-60d7-4774-9fa4-fa72ba95629e admin admin] PATH_INFO: `/v3/users/12696013c1d64937a50db9fc36e548e9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:40.513485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef2b1f-60d7-4774-9fa4-fa72ba95629e admin admin] RBAC: Authorizing `identity:delete_user(user_id=12696013c1d64937a50db9fc36e548e9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:40.513981 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:40.515735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef2b1f-60d7-4774-9fa4-fa72ba95629e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:40.596715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-feef2b1f-60d7-4774-9fa4-fa72ba95629e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12696013c1d64937a50db9fc36e548e9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:40.611356 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1c59691cc147a69ccb3014f74ec98e', 'request_id': 'req-a9b75976-1a00-4196-96cb-fd6b83ea1379'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:40.617149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-feef2b1f-60d7-4774-9fa4-fa72ba95629e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12696013c1d64937a50db9fc36e548e9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:40.618335 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-feef2b1f-60d7-4774-9fa4-fa72ba95629e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12696013c1d64937a50db9fc36e548e9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:40.620482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1c59691cc147a69ccb3014f74ec98e', 'request_id': 'req-a9b75976-1a00-4196-96cb-fd6b83ea1379'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:40.631061 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1c59691cc147a69ccb3014f74ec98e', 'request_id': 'req-a9b75976-1a00-4196-96cb-fd6b83ea1379'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:40.631618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-feef2b1f-60d7-4774-9fa4-fa72ba95629e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12696013c1d64937a50db9fc36e548e9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:40.647829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-feef2b1f-60d7-4774-9fa4-fa72ba95629e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12696013c1d64937a50db9fc36e548e9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:40.648410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] The token cache is being invalidate because project 7d1c59691cc147a69ccb3014f74ec98e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:40.648966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:40.650053 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a9b75976-1a00-4196-96cb-fd6b83ea1379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:40.654715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 872/1764] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:40 2022] DELETE /identity/v3/projects/7d1c59691cc147a69ccb3014f74ec98e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:40.659652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045899c-544b-4f91-87f8-7d1b431541ff None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:40.660166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045899c-544b-4f91-87f8-7d1b431541ff None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:40.660679 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 893/1765] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:40 2022] DELETE /identity/v3/users/12696013c1d64937a50db9fc36e548e9 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:40.665583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045899c-544b-4f91-87f8-7d1b431541ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:40.668515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890b58d-9b04-4723-8d29-2438c07a5d7a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:40.669279 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890b58d-9b04-4723-8d29-2438c07a5d7a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:40.670098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890b58d-9b04-4723-8d29-2438c07a5d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:40.691057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c045899c-544b-4f91-87f8-7d1b431541ff None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:40.692290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c045899c-544b-4f91-87f8-7d1b431541ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:40.695998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-1890b58d-9b04-4723-8d29-2438c07a5d7a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:40.698100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1890b58d-9b04-4723-8d29-2438c07a5d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:40.751250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:40.767904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 873/1766] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:40.775971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 894/1767] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:40.776470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec7cc6-8227-4bd1-a9ee-cb7d68fd9eb1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:40.776766 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec7cc6-8227-4bd1-a9ee-cb7d68fd9eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:40.777048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec7cc6-8227-4bd1-a9ee-cb7d68fd9eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:40.787408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890b58d-9b04-4723-8d29-2438c07a5d7a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:40.789260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1890b58d-9b04-4723-8d29-2438c07a5d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:40.797443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-29ec7cc6-8227-4bd1-a9ee-cb7d68fd9eb1 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:40.798796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-29ec7cc6-8227-4bd1-a9ee-cb7d68fd9eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:40.823009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 874/1768] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:26:40.826168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec7cc6-8227-4bd1-a9ee-cb7d68fd9eb1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:40.836611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec7cc6-8227-4bd1-a9ee-cb7d68fd9eb1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:40.843511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bb4f8-c450-4cdf-bf50-27c2cde73620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:40.844428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bb4f8-c450-4cdf-bf50-27c2cde73620 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:40.844718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bb4f8-c450-4cdf-bf50-27c2cde73620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:40.845002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bb4f8-c450-4cdf-bf50-27c2cde73620 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:40.845750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bb4f8-c450-4cdf-bf50-27c2cde73620 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:26:40.846069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bb4f8-c450-4cdf-bf50-27c2cde73620 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:40.848535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bb4f8-c450-4cdf-bf50-27c2cde73620 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:40.862391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 875/1769] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:26:40.867285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bb4f8-c450-4cdf-bf50-27c2cde73620 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:40.890195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bb4f8-c450-4cdf-bf50-27c2cde73620 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:40.896590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7410dbf-39b3-4dab-962f-36ce23b2ce09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:40.899423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7410dbf-39b3-4dab-962f-36ce23b2ce09 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:40.899910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7410dbf-39b3-4dab-962f-36ce23b2ce09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:40.900399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7410dbf-39b3-4dab-962f-36ce23b2ce09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:40.901286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7410dbf-39b3-4dab-962f-36ce23b2ce09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:40.903554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7410dbf-39b3-4dab-962f-36ce23b2ce09 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:40.908670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890b58d-9b04-4723-8d29-2438c07a5d7a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:40.909516 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:40.916594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e184ace-1ea9-407f-8ec5-ddfc55e71cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:40.917951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e184ace-1ea9-407f-8ec5-ddfc55e71cfd service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:40.918790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e184ace-1ea9-407f-8ec5-ddfc55e71cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:40.919337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e184ace-1ea9-407f-8ec5-ddfc55e71cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:40.943250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e184ace-1ea9-407f-8ec5-ddfc55e71cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:40.943539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e184ace-1ea9-407f-8ec5-ddfc55e71cfd service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:40.959694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 876/1770] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:26:40 2022] POST /identity/v3/projects => generated 365 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:40.963033 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 895/1771] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:26:40.967803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7410dbf-39b3-4dab-962f-36ce23b2ce09 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:41.072234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7410dbf-39b3-4dab-962f-36ce23b2ce09 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:41.082865 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfdaf8-92fc-4b46-9784-978ecabbc7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:41.084013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfdaf8-92fc-4b46-9784-978ecabbc7ef admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:41.084512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfdaf8-92fc-4b46-9784-978ecabbc7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:41.085204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfdaf8-92fc-4b46-9784-978ecabbc7ef admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:41.086450 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbfdaf8-92fc-4b46-9784-978ecabbc7ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:41.090250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbfdaf8-92fc-4b46-9784-978ecabbc7ef admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:41.134263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 877/1772] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:26:40 2022] POST /identity/v3/users => generated 283 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:41.135553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e184ace-1ea9-407f-8ec5-ddfc55e71cfd service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:41.149831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e184ace-1ea9-407f-8ec5-ddfc55e71cfd service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:41.154980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0906be-9737-4e45-a6b3-a7b6762a0c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:41.155906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0906be-9737-4e45-a6b3-a7b6762a0c45 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:41.156893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0906be-9737-4e45-a6b3-a7b6762a0c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:41.157306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0906be-9737-4e45-a6b3-a7b6762a0c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:41.158677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0906be-9737-4e45-a6b3-a7b6762a0c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:41.160321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0906be-9737-4e45-a6b3-a7b6762a0c45 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:41.181909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 896/1773] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:41.189500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfdaf8-92fc-4b46-9784-978ecabbc7ef admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:41.213306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbfdaf8-92fc-4b46-9784-978ecabbc7ef admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:41.227507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a74742-23f3-4610-968b-e736cb96ea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:41.228890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a74742-23f3-4610-968b-e736cb96ea2a admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:41.229494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a74742-23f3-4610-968b-e736cb96ea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:41.230015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a74742-23f3-4610-968b-e736cb96ea2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:41.231496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a74742-23f3-4610-968b-e736cb96ea2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:41.233387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a74742-23f3-4610-968b-e736cb96ea2a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:41.252634 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 878/1774] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:41.263467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0906be-9737-4e45-a6b3-a7b6762a0c45 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:41.283342 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0906be-9737-4e45-a6b3-a7b6762a0c45 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:41.293787 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cfbed-643b-4134-a35d-03131119dbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:41.294872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cfbed-643b-4134-a35d-03131119dbcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:41.295385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cfbed-643b-4134-a35d-03131119dbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:41.295912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cfbed-643b-4134-a35d-03131119dbcc admin admin] PATH_INFO: `/v3/projects/9b236b3e528d4e2d842a7d5f5cab2f50/users/c64d6f42597d4ab591577dff4c287703/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:41.333885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cfbed-643b-4134-a35d-03131119dbcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b236b3e528d4e2d842a7d5f5cab2f50, user_id=c64d6f42597d4ab591577dff4c287703, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:41.340703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cfbed-643b-4134-a35d-03131119dbcc admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:41.368674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 897/1775] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:41 2022] PUT /identity/v3/projects/9b236b3e528d4e2d842a7d5f5cab2f50/users/c64d6f42597d4ab591577dff4c287703/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:41.390641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf70a-556f-49f4-b297-599af526811c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:41.391040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf70a-556f-49f4-b297-599af526811c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:41.391524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaf70a-556f-49f4-b297-599af526811c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:41.435281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cfbed-643b-4134-a35d-03131119dbcc admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:41.439108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-942cfbed-643b-4134-a35d-03131119dbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:41.477355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-47aaf70a-556f-49f4-b297-599af526811c None None] MFA Rules not processed for user `c64d6f42597d4ab591577dff4c287703`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:41.477355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-47aaf70a-556f-49f4-b297-599af526811c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:41.623838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cfbed-643b-4134-a35d-03131119dbcc admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:41.635025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:41.636747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:41.637063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:41.637560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] PATH_INFO: `/v3/projects/28f0e3883e9b4eca93f94a271e332ff7` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:41.642980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 879/1776] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:41.643192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f0e3883e9b4eca93f94a271e332ff7)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:41.646617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:41.650372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf70a-556f-49f4-b297-599af526811c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:41.686209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaf70a-556f-49f4-b297-599af526811c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:41.699258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522e39b-1161-46ad-89a7-859399a6f378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:41.700240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522e39b-1161-46ad-89a7-859399a6f378 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:41.700691 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522e39b-1161-46ad-89a7-859399a6f378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:41.701205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e522e39b-1161-46ad-89a7-859399a6f378 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:41.703794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522e39b-1161-46ad-89a7-859399a6f378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:41.709577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e522e39b-1161-46ad-89a7-859399a6f378 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:41.775838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f0e3883e9b4eca93f94a271e332ff7', 'request_id': 'req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:41.791160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f0e3883e9b4eca93f94a271e332ff7', 'request_id': 'req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:41.795776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:41.798927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 880/1777] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:26:41 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:26:41.803727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522e39b-1161-46ad-89a7-859399a6f378 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:41.811080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f0e3883e9b4eca93f94a271e332ff7', 'request_id': 'req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:41.836779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] The token cache is being invalidate because project 28f0e3883e9b4eca93f94a271e332ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:41.837254 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:41.838999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:41.844453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 898/1778] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:41 2022] DELETE /identity/v3/projects/28f0e3883e9b4eca93f94a271e332ff7 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:26:41.850553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:41.853556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:41.865723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:41.914348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e522e39b-1161-46ad-89a7-859399a6f378 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:41.923583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae4e09-670b-4aec-ae36-a17d2718d600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:41.924494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae4e09-670b-4aec-ae36-a17d2718d600 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:41.924839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae4e09-670b-4aec-ae36-a17d2718d600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:41.925129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fae4e09-670b-4aec-ae36-a17d2718d600 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:41.925893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae4e09-670b-4aec-ae36-a17d2718d600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:41.929382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fae4e09-670b-4aec-ae36-a17d2718d600 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:41.980957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 881/1779] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:26:41 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:41.987121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae4e09-670b-4aec-ae36-a17d2718d600 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:41.988979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0fae4e09-670b-4aec-ae36-a17d2718d600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:41.994470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c43d4-dbcd-4995-bbb1-959cd8bb2c20 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:42.009377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fae4e09-670b-4aec-ae36-a17d2718d600 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:42.010913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:42.012488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:42.012984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:42.013598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] PATH_INFO: `/v3/users/5fac501d481248eab88d3871a116aaad` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:42.028352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fac501d481248eab88d3871a116aaad)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:42.031509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:42.032377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f3d2d-c99f-4c43-897e-a6f49281c122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:42.033493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f3d2d-c99f-4c43-897e-a6f49281c122 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:42.034015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f3d2d-c99f-4c43-897e-a6f49281c122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:42.034598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f3d2d-c99f-4c43-897e-a6f49281c122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:42.035971 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f3d2d-c99f-4c43-897e-a6f49281c122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:42.045610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f3d2d-c99f-4c43-897e-a6f49281c122 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:42.095260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 882/1780] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:42.097818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f3d2d-c99f-4c43-897e-a6f49281c122 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:42.141733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f3d2d-c99f-4c43-897e-a6f49281c122 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:42.168572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935c31e-77fd-4a62-bafc-5e5d97260e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:42.175082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935c31e-77fd-4a62-bafc-5e5d97260e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:42.176368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935c31e-77fd-4a62-bafc-5e5d97260e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:42.179393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935c31e-77fd-4a62-bafc-5e5d97260e94 admin admin] PATH_INFO: `/v3/projects/3dc8ade0a5294f71ae2b924822c6fb91/users/ae6f59fbc44840998271aa6286552b08/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:42.198026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fac501d481248eab88d3871a116aaad'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:42.245334 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935c31e-77fd-4a62-bafc-5e5d97260e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc8ade0a5294f71ae2b924822c6fb91, user_id=ae6f59fbc44840998271aa6286552b08, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:42.247235 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fac501d481248eab88d3871a116aaad'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:42.248228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fac501d481248eab88d3871a116aaad'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:42.248408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935c31e-77fd-4a62-bafc-5e5d97260e94 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:42.265252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fac501d481248eab88d3871a116aaad'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:42.280962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 883/1781] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:42 2022] PUT /identity/v3/projects/3dc8ade0a5294f71ae2b924822c6fb91/users/ae6f59fbc44840998271aa6286552b08/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:42.284012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935c31e-77fd-4a62-bafc-5e5d97260e94 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:42.285997 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fac501d481248eab88d3871a116aaad'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:42.317423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 899/1782] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:41 2022] DELETE /identity/v3/users/5fac501d481248eab88d3871a116aaad => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:26:42.442815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935c31e-77fd-4a62-bafc-5e5d97260e94 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:42.589145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89c401-95bd-4eeb-a2d5-6913c8b4465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:42.590753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89c401-95bd-4eeb-a2d5-6913c8b4465b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:42.591580 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89c401-95bd-4eeb-a2d5-6913c8b4465b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:42.592275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89c401-95bd-4eeb-a2d5-6913c8b4465b admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:42.595315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89c401-95bd-4eeb-a2d5-6913c8b4465b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:42.600480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89c401-95bd-4eeb-a2d5-6913c8b4465b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:42.628380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 884/1783] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:26:42.633073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:42.636152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:42.676349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce5e389-3e71-405c-96d1-26148410314d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:42.685936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f9f90-2a36-40c7-9733-a466daf9e923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:42.688919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f9f90-2a36-40c7-9733-a466daf9e923 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:42.689684 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f9f90-2a36-40c7-9733-a466daf9e923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:42.689958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f9f90-2a36-40c7-9733-a466daf9e923 admin admin] PATH_INFO: `/v3/projects/3dc8ade0a5294f71ae2b924822c6fb91/users/ae6f59fbc44840998271aa6286552b08/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:42.693887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f9f90-2a36-40c7-9733-a466daf9e923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc8ade0a5294f71ae2b924822c6fb91, user_id=ae6f59fbc44840998271aa6286552b08, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:42.700604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f9f90-2a36-40c7-9733-a466daf9e923 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:42.738645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 900/1784] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:42 2022] PUT /identity/v3/projects/3dc8ade0a5294f71ae2b924822c6fb91/users/ae6f59fbc44840998271aa6286552b08/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:42.749156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fe9bf-3e05-47f2-83d9-a0744d4cfadf None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:42.750551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fe9bf-3e05-47f2-83d9-a0744d4cfadf None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:42.752402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32fe9bf-3e05-47f2-83d9-a0744d4cfadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:42.825677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a32fe9bf-3e05-47f2-83d9-a0744d4cfadf None None] MFA Rules not processed for user `ae6f59fbc44840998271aa6286552b08`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:42.827426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a32fe9bf-3e05-47f2-83d9-a0744d4cfadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:42.850473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:42.986914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 885/1785] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:43.001395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f9f90-2a36-40c7-9733-a466daf9e923 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:43.010783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d47f9f90-2a36-40c7-9733-a466daf9e923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:43.127544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f9f90-2a36-40c7-9733-a466daf9e923 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:43.134480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040a852-88b5-4ada-b84d-7977355f1a04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:43.135427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040a852-88b5-4ada-b84d-7977355f1a04 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:43.135856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040a852-88b5-4ada-b84d-7977355f1a04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:43.136262 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040a852-88b5-4ada-b84d-7977355f1a04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:43.146925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:43.156817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040a852-88b5-4ada-b84d-7977355f1a04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:43.160479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040a852-88b5-4ada-b84d-7977355f1a04 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:43.188350 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 901/1786] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:43.309372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fe9bf-3e05-47f2-83d9-a0744d4cfadf None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:43.324915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32fe9bf-3e05-47f2-83d9-a0744d4cfadf None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:43.330140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748135d-4300-49b6-815b-32d20a9cd957 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:43.330944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748135d-4300-49b6-815b-32d20a9cd957 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:43.331373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748135d-4300-49b6-815b-32d20a9cd957 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:43.335549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748135d-4300-49b6-815b-32d20a9cd957 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:43.351115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d748135d-4300-49b6-815b-32d20a9cd957 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:43.542190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748135d-4300-49b6-815b-32d20a9cd957 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:43.544820 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d748135d-4300-49b6-815b-32d20a9cd957 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:43.561788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 886/1787] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:43.919265 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040a852-88b5-4ada-b84d-7977355f1a04 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:43.922406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b040a852-88b5-4ada-b84d-7977355f1a04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:44.049263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040a852-88b5-4ada-b84d-7977355f1a04 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:44.056114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:44.057022 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:44.057295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:44.057761 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] PATH_INFO: `/v3/projects/b3acd4d70ed94f3b9a473db433ccdbc1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:44.059304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3acd4d70ed94f3b9a473db433ccdbc1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:44.063284 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:44.168503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3acd4d70ed94f3b9a473db433ccdbc1', 'request_id': 'req-8306a660-873e-4716-a42d-46aeddf1a368'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:44.185857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3acd4d70ed94f3b9a473db433ccdbc1', 'request_id': 'req-8306a660-873e-4716-a42d-46aeddf1a368'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:44.211108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3acd4d70ed94f3b9a473db433ccdbc1', 'request_id': 'req-8306a660-873e-4716-a42d-46aeddf1a368'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:44.239040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] The token cache is being invalidate because project b3acd4d70ed94f3b9a473db433ccdbc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:44.239644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:44.240579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-8306a660-873e-4716-a42d-46aeddf1a368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:44.244119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 902/1788] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:43 2022] DELETE /identity/v3/projects/b3acd4d70ed94f3b9a473db433ccdbc1 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:44.250781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d1509-fb46-4e92-a5b3-f97566ede4e9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:44.251379 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d1509-fb46-4e92-a5b3-f97566ede4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:44.251839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273d1509-fb46-4e92-a5b3-f97566ede4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:44.286200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-273d1509-fb46-4e92-a5b3-f97566ede4e9 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:44.288853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-273d1509-fb46-4e92-a5b3-f97566ede4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:44.404599 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 887/1789] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:44.420466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62822d-c0a5-429c-919c-cd8761e1dc8d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:44.420740 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62822d-c0a5-429c-919c-cd8761e1dc8d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:44.421033 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62822d-c0a5-429c-919c-cd8761e1dc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:44.442820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ba62822d-c0a5-429c-919c-cd8761e1dc8d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:44.443655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ba62822d-c0a5-429c-919c-cd8761e1dc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:44.460336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 903/1790] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:44.467135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d1509-fb46-4e92-a5b3-f97566ede4e9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:44.487023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273d1509-fb46-4e92-a5b3-f97566ede4e9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:44.496118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24e9d7-38c7-43bf-860e-9153e11b9300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:44.497645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24e9d7-38c7-43bf-860e-9153e11b9300 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:44.498047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24e9d7-38c7-43bf-860e-9153e11b9300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:44.498635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24e9d7-38c7-43bf-860e-9153e11b9300 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:44.500532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24e9d7-38c7-43bf-860e-9153e11b9300 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:26:44.501068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24e9d7-38c7-43bf-860e-9153e11b9300 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:44.504495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24e9d7-38c7-43bf-860e-9153e11b9300 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:44.538074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 888/1791] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:26:44.543955 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62822d-c0a5-429c-919c-cd8761e1dc8d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:44.563000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62822d-c0a5-429c-919c-cd8761e1dc8d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:44.571867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580031b-c8ff-4c49-9b46-f7561b4d06b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:44.573187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580031b-c8ff-4c49-9b46-f7561b4d06b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:44.574837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580031b-c8ff-4c49-9b46-f7561b4d06b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:44.574837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f580031b-c8ff-4c49-9b46-f7561b4d06b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:44.576215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f580031b-c8ff-4c49-9b46-f7561b4d06b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:44.580266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f580031b-c8ff-4c49-9b46-f7561b4d06b7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:44.626332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 904/1792] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:26:44 2022] POST /identity/v3/projects => generated 391 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:44.630503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24e9d7-38c7-43bf-860e-9153e11b9300 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:44.646302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24e9d7-38c7-43bf-860e-9153e11b9300 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:44.696876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:44.721414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5031f-f439-49fe-8627-018c936690ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:44.722281 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5031f-f439-49fe-8627-018c936690ee admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:44.722571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5031f-f439-49fe-8627-018c936690ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:44.722892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5031f-f439-49fe-8627-018c936690ee admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:44.723648 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5031f-f439-49fe-8627-018c936690ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:44.726352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5031f-f439-49fe-8627-018c936690ee admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:44.762937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 889/1793] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:26:44 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:44.769438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580031b-c8ff-4c49-9b46-f7561b4d06b7 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:44.803073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f580031b-c8ff-4c49-9b46-f7561b4d06b7 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:44.813766 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d122470-1daa-446e-8bc0-c8a8a1ecc43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:44.814895 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d122470-1daa-446e-8bc0-c8a8a1ecc43f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:44.815331 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d122470-1daa-446e-8bc0-c8a8a1ecc43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:44.815953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d122470-1daa-446e-8bc0-c8a8a1ecc43f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:44.817192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d122470-1daa-446e-8bc0-c8a8a1ecc43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:44.821132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d122470-1daa-446e-8bc0-c8a8a1ecc43f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:44.846975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 905/1794] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:44.851410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5031f-f439-49fe-8627-018c936690ee admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:44.870919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5031f-f439-49fe-8627-018c936690ee admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:44.880845 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec112d55-6c31-4319-94d1-d4e88d9612e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:44.882031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112d55-6c31-4319-94d1-d4e88d9612e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:44.882605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112d55-6c31-4319-94d1-d4e88d9612e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:44.883157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec112d55-6c31-4319-94d1-d4e88d9612e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:44.884821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec112d55-6c31-4319-94d1-d4e88d9612e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:44.904496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec112d55-6c31-4319-94d1-d4e88d9612e3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:44.934663 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 890/1795] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:44.935742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d122470-1daa-446e-8bc0-c8a8a1ecc43f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:44.954383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d122470-1daa-446e-8bc0-c8a8a1ecc43f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:44.963215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14fc36-3c7d-4c42-b053-85ea59be6fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:44.965096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14fc36-3c7d-4c42-b053-85ea59be6fae admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:44.965850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14fc36-3c7d-4c42-b053-85ea59be6fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:44.966336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14fc36-3c7d-4c42-b053-85ea59be6fae admin admin] PATH_INFO: `/v3/projects/db98840783b642c8884d04404816013f/users/02f41d43ce4547d88edd1c4483eb109e/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:44.997321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14fc36-3c7d-4c42-b053-85ea59be6fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=db98840783b642c8884d04404816013f, user_id=02f41d43ce4547d88edd1c4483eb109e, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:45.001752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14fc36-3c7d-4c42-b053-85ea59be6fae admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:45.033045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 906/1796] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:44 2022] PUT /identity/v3/projects/db98840783b642c8884d04404816013f/users/02f41d43ce4547d88edd1c4483eb109e/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:45.053848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1af98-65ef-4fb1-a956-54781f6552af None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:45.054370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1af98-65ef-4fb1-a956-54781f6552af None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:45.054785 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1af98-65ef-4fb1-a956-54781f6552af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:45.146216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f3a1af98-65ef-4fb1-a956-54781f6552af None None] MFA Rules not processed for user `02f41d43ce4547d88edd1c4483eb109e`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:45.149354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f3a1af98-65ef-4fb1-a956-54781f6552af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:45.279717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 891/1797] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:45.285482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14fc36-3c7d-4c42-b053-85ea59be6fae admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:45.324189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:45.430108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14fc36-3c7d-4c42-b053-85ea59be6fae admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:45.439914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda087d6-e23a-46ab-837d-db89a4e3e2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:45.440962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda087d6-e23a-46ab-837d-db89a4e3e2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:45.441619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda087d6-e23a-46ab-837d-db89a4e3e2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:45.442115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda087d6-e23a-46ab-837d-db89a4e3e2b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:45.443763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda087d6-e23a-46ab-837d-db89a4e3e2b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:45.447671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda087d6-e23a-46ab-837d-db89a4e3e2b1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:45.528282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 907/1798] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:26:45 2022] POST /identity/v3/projects => generated 393 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:26:45.533602 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1af98-65ef-4fb1-a956-54781f6552af None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:45.556703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1af98-65ef-4fb1-a956-54781f6552af None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:45.676018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b17ae7-ee2a-4ca6-b788-be6033f8209f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:45.678877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b17ae7-ee2a-4ca6-b788-be6033f8209f admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:45.679322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b17ae7-ee2a-4ca6-b788-be6033f8209f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:45.679717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b17ae7-ee2a-4ca6-b788-be6033f8209f admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:45.682421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b17ae7-ee2a-4ca6-b788-be6033f8209f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:45.688474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b17ae7-ee2a-4ca6-b788-be6033f8209f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:45.738998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 892/1799] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:26:45 2022] POST /identity/v3/users => generated 303 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:26:45.745014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda087d6-e23a-46ab-837d-db89a4e3e2b1 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:45.757554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda087d6-e23a-46ab-837d-db89a4e3e2b1 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:45.762853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a640ce-81e6-4dea-9766-883a97b3468d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:45.763689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a640ce-81e6-4dea-9766-883a97b3468d admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:45.763989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a640ce-81e6-4dea-9766-883a97b3468d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:45.764280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a640ce-81e6-4dea-9766-883a97b3468d admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:45.765186 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a640ce-81e6-4dea-9766-883a97b3468d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:45.768284 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a640ce-81e6-4dea-9766-883a97b3468d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:45.799049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 908/1800] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:45.800767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b17ae7-ee2a-4ca6-b788-be6033f8209f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:45.813037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b17ae7-ee2a-4ca6-b788-be6033f8209f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:45.819141 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c33ed-732d-4b03-8d1f-d7ab2a1cdaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:45.820771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c33ed-732d-4b03-8d1f-d7ab2a1cdaea admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:45.821332 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c33ed-732d-4b03-8d1f-d7ab2a1cdaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:45.821893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c33ed-732d-4b03-8d1f-d7ab2a1cdaea admin admin] PATH_INFO: `/v3/projects/fe2a3abe9b0d44d8872c877fd710e837/users/47c455450ecb44149927fba98453c150/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:45.880929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c33ed-732d-4b03-8d1f-d7ab2a1cdaea admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2a3abe9b0d44d8872c877fd710e837, user_id=47c455450ecb44149927fba98453c150, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:45.887971 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c33ed-732d-4b03-8d1f-d7ab2a1cdaea admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:45.920447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 893/1801] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:45 2022] PUT /identity/v3/projects/fe2a3abe9b0d44d8872c877fd710e837/users/47c455450ecb44149927fba98453c150/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:45.924746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a640ce-81e6-4dea-9766-883a97b3468d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:45.946985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a640ce-81e6-4dea-9766-883a97b3468d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:46.040891 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed188c-e4ae-47fe-923b-5110045f554f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:46.041919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed188c-e4ae-47fe-923b-5110045f554f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:46.042164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed188c-e4ae-47fe-923b-5110045f554f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:46.042454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed188c-e4ae-47fe-923b-5110045f554f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:46.043223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed188c-e4ae-47fe-923b-5110045f554f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:46.044900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed188c-e4ae-47fe-923b-5110045f554f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:46.059798 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 909/1802] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:46.064118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c33ed-732d-4b03-8d1f-d7ab2a1cdaea admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:46.075130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c33ed-732d-4b03-8d1f-d7ab2a1cdaea admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:46.080876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8199200-82c7-42a7-a329-19261d83caa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:46.081786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8199200-82c7-42a7-a329-19261d83caa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:46.082071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8199200-82c7-42a7-a329-19261d83caa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:46.082347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8199200-82c7-42a7-a329-19261d83caa8 admin admin] PATH_INFO: `/v3/projects/fe2a3abe9b0d44d8872c877fd710e837/users/47c455450ecb44149927fba98453c150/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:46.084496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8199200-82c7-42a7-a329-19261d83caa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2a3abe9b0d44d8872c877fd710e837, user_id=47c455450ecb44149927fba98453c150, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:46.087477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8199200-82c7-42a7-a329-19261d83caa8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:46.095460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:46.101537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 894/1803] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:46 2022] PUT /identity/v3/projects/fe2a3abe9b0d44d8872c877fd710e837/users/47c455450ecb44149927fba98453c150/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:46.114863 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7352fd-a6e9-4af1-b59c-473dbad20c6d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:46.115152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7352fd-a6e9-4af1-b59c-473dbad20c6d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:46.115431 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7352fd-a6e9-4af1-b59c-473dbad20c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:46.149219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3b7352fd-a6e9-4af1-b59c-473dbad20c6d None None] MFA Rules not processed for user `47c455450ecb44149927fba98453c150`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:46.150457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3b7352fd-a6e9-4af1-b59c-473dbad20c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:46.240187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 910/1804] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:46.259679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8199200-82c7-42a7-a329-19261d83caa8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:46.261499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d8199200-82c7-42a7-a329-19261d83caa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:46.340192 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8199200-82c7-42a7-a329-19261d83caa8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:46.350263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74c10d-65b8-450f-8b42-b3d3cc215484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:46.351486 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c10d-65b8-450f-8b42-b3d3cc215484 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:46.352008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c10d-65b8-450f-8b42-b3d3cc215484 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:46.352521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74c10d-65b8-450f-8b42-b3d3cc215484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:46.397257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74c10d-65b8-450f-8b42-b3d3cc215484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:46.398895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74c10d-65b8-450f-8b42-b3d3cc215484 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:46.411914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 895/1805] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:26:46.484255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7352fd-a6e9-4af1-b59c-473dbad20c6d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:46.486423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3b7352fd-a6e9-4af1-b59c-473dbad20c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:46.502150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7352fd-a6e9-4af1-b59c-473dbad20c6d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:46.515203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fff5c2-1ad5-4d5c-bf08-f89685d4c95f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:46.517294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fff5c2-1ad5-4d5c-bf08-f89685d4c95f service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:46.517601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fff5c2-1ad5-4d5c-bf08-f89685d4c95f service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:46.517904 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fff5c2-1ad5-4d5c-bf08-f89685d4c95f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:46.535640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:46.598338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fff5c2-1ad5-4d5c-bf08-f89685d4c95f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:46.599842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fff5c2-1ad5-4d5c-bf08-f89685d4c95f service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:46.611088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 911/1806] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:53.572312 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74c10d-65b8-450f-8b42-b3d3cc215484 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:53.696333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74c10d-65b8-450f-8b42-b3d3cc215484 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:53.702474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:53.704371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:53.704955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:53.705423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] PATH_INFO: `/v3/users/02f41d43ce4547d88edd1c4483eb109e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:53.707944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=02f41d43ce4547d88edd1c4483eb109e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:53.710822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:53.821143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02f41d43ce4547d88edd1c4483eb109e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:53.841613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02f41d43ce4547d88edd1c4483eb109e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:53.858509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02f41d43ce4547d88edd1c4483eb109e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:53.888396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02f41d43ce4547d88edd1c4483eb109e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:53.889336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02f41d43ce4547d88edd1c4483eb109e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:53.897414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 896/1807] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:53 2022] DELETE /identity/v3/users/02f41d43ce4547d88edd1c4483eb109e => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:53.903431 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf96dcc-0c4a-4bd2-8392-00f6facde58c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:53.905070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf96dcc-0c4a-4bd2-8392-00f6facde58c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:53.907140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf96dcc-0c4a-4bd2-8392-00f6facde58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:53.930039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-eaf96dcc-0c4a-4bd2-8392-00f6facde58c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:53.931642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eaf96dcc-0c4a-4bd2-8392-00f6facde58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:54.034640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 912/1808] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:54.051114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:54.052909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:54.103976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:54.156645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9670b-0a33-4f8d-8275-0fcebcbfb8cc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:54.161555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269aaad-2e29-4279-8861-420bf23ba89f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:54.162308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269aaad-2e29-4279-8861-420bf23ba89f service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:54.162626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269aaad-2e29-4279-8861-420bf23ba89f service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:54.163927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269aaad-2e29-4279-8861-420bf23ba89f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:54.176157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269aaad-2e29-4279-8861-420bf23ba89f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:54.177665 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269aaad-2e29-4279-8861-420bf23ba89f service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:54.198749 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 897/1809] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:54.588522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf96dcc-0c4a-4bd2-8392-00f6facde58c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:54.591170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eaf96dcc-0c4a-4bd2-8392-00f6facde58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:54.591710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7abc5-eca1-40b8-8ca9-0aff8767daea None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:54.592439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7abc5-eca1-40b8-8ca9-0aff8767daea None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:54.592981 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c7abc5-eca1-40b8-8ca9-0aff8767daea None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:54.598392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 898/1810] 10.222.0.64 () {54 vars in 921 bytes} [Wed Jul 20 08:26:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 08:26:54.625492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641700-f382-45e1-ba46-ea086145381e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:54.627522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641700-f382-45e1-ba46-ea086145381e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:54.631504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641700-f382-45e1-ba46-ea086145381e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:54.646913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf96dcc-0c4a-4bd2-8392-00f6facde58c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:54.671610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:54.673758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:54.674157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1e641700-f382-45e1-ba46-ea086145381e None None] MFA Rules not processed for user `928054a577d14522bd0f00417f4f8c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:54.674322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:54.674763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] PATH_INFO: `/v3/projects/db98840783b642c8884d04404816013f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:54.675530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1e641700-f382-45e1-ba46-ea086145381e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:54.677066 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db98840783b642c8884d04404816013f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:54.682813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:54.832545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 899/1811] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Jul 20 08:26:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:54.853130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db98840783b642c8884d04404816013f', 'request_id': 'req-467720bb-229f-40c8-80b7-f699d6dcd9e8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:54.872190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db98840783b642c8884d04404816013f', 'request_id': 'req-467720bb-229f-40c8-80b7-f699d6dcd9e8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:54.892882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db98840783b642c8884d04404816013f', 'request_id': 'req-467720bb-229f-40c8-80b7-f699d6dcd9e8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:54.930768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e641700-f382-45e1-ba46-ea086145381e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:54.934522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1e641700-f382-45e1-ba46-ea086145381e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:54.940712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] The token cache is being invalidate because project db98840783b642c8884d04404816013f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:54.941322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:54.943502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:54.969469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 913/1812] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:54 2022] DELETE /identity/v3/projects/db98840783b642c8884d04404816013f => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:26:54.982374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:54.985801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:54.991813 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e641700-f382-45e1-ba46-ea086145381e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:55.124616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:55.191206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467720bb-229f-40c8-80b7-f699d6dcd9e8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:55.193505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c1aec-1422-4147-9266-576e3069cda6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:55.197547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c1aec-1422-4147-9266-576e3069cda6 service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:55.197793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c1aec-1422-4147-9266-576e3069cda6 service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:55.198082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c1aec-1422-4147-9266-576e3069cda6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:55.202406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:55.205280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:55.205457 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-269c1aec-1422-4147-9266-576e3069cda6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:55.206507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:55.207169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] PATH_INFO: `/v3/users/47c455450ecb44149927fba98453c150` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:55.210226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c455450ecb44149927fba98453c150)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:55.216403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:55.231470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c1aec-1422-4147-9266-576e3069cda6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:55.233571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269c1aec-1422-4147-9266-576e3069cda6 service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:55.274680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 900/1813] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:55.361798 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c455450ecb44149927fba98453c150'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:55.382836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c455450ecb44149927fba98453c150'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:55.388019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c455450ecb44149927fba98453c150'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:55.424502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c455450ecb44149927fba98453c150'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:55.464518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c455450ecb44149927fba98453c150'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:55.482630 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 914/1814] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:54 2022] DELETE /identity/v3/users/47c455450ecb44149927fba98453c150 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:55.981561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c1aec-1422-4147-9266-576e3069cda6 service placement] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:55.993815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c1aec-1422-4147-9266-576e3069cda6 service placement] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:56.096134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:56.127105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:56.130689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:56.131610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:56.131773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:56.132070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:56.132230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] PATH_INFO: `/v3/projects/fe2a3abe9b0d44d8872c877fd710e837` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:56.133677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2a3abe9b0d44d8872c877fd710e837)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:56.135744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:56.162085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bc38a-70ba-4c0d-9230-ca4a70db871d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:56.173857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:56.175401 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:56.175700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:56.175987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] PATH_INFO: `/v3/users/b99cc3ebd1674da8936964a837c628f9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:56.177429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b99cc3ebd1674da8936964a837c628f9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:56.181343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:56.234864 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2a3abe9b0d44d8872c877fd710e837', 'request_id': 'req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.251909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2a3abe9b0d44d8872c877fd710e837', 'request_id': 'req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.260760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2a3abe9b0d44d8872c877fd710e837', 'request_id': 'req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.264820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b99cc3ebd1674da8936964a837c628f9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.282841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] The token cache is being invalidate because project fe2a3abe9b0d44d8872c877fd710e837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:56.283149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.283908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfe0fa91-8a2b-489b-bc70-b2b88b5861f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.286155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b99cc3ebd1674da8936964a837c628f9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.290119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 901/1815] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:55 2022] DELETE /identity/v3/projects/fe2a3abe9b0d44d8872c877fd710e837 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:26:56.291038 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b99cc3ebd1674da8936964a837c628f9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.292861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758a386-7ac9-4490-8ea8-7a7c0dc22e53 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:56.293165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758a386-7ac9-4490-8ea8-7a7c0dc22e53 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:56.295106 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758a386-7ac9-4490-8ea8-7a7c0dc22e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:56.306283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b99cc3ebd1674da8936964a837c628f9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.320812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e758a386-7ac9-4490-8ea8-7a7c0dc22e53 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:56.321392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e758a386-7ac9-4490-8ea8-7a7c0dc22e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:56.331292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b99cc3ebd1674da8936964a837c628f9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.346221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 915/1816] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:56 2022] DELETE /identity/v3/users/b99cc3ebd1674da8936964a837c628f9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:56.425162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:56.427483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:56.474356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:56.476564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 902/1817] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:56.485954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d6b8b-beeb-4a1f-ab78-5794d192928f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:56.485954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d6b8b-beeb-4a1f-ab78-5794d192928f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:56.486357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d6b8b-beeb-4a1f-ab78-5794d192928f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:56.509153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4c4d6b8b-beeb-4a1f-ab78-5794d192928f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:56.510465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4c4d6b8b-beeb-4a1f-ab78-5794d192928f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:56.598554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed932819-4b7e-402b-9e7e-b96bcfbd4f20 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:56.601933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 903/1818] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:26:56.603080 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d6b8b-beeb-4a1f-ab78-5794d192928f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:56.610325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff25c58-9448-4779-ae21-11047ea59868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:56.613086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff25c58-9448-4779-ae21-11047ea59868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:56.615233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d6b8b-beeb-4a1f-ab78-5794d192928f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:56.618763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff25c58-9448-4779-ae21-11047ea59868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:56.619169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff25c58-9448-4779-ae21-11047ea59868 admin admin] PATH_INFO: `/v3/users/c64d6f42597d4ab591577dff4c287703` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:56.623473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f49d1-7470-4adf-9acc-8f40cb012e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:56.624176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f49d1-7470-4adf-9acc-8f40cb012e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:56.624525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f49d1-7470-4adf-9acc-8f40cb012e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:56.624846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f49d1-7470-4adf-9acc-8f40cb012e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:56.626003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f49d1-7470-4adf-9acc-8f40cb012e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:26:56.626781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff25c58-9448-4779-ae21-11047ea59868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64d6f42597d4ab591577dff4c287703)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:56.629056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff25c58-9448-4779-ae21-11047ea59868 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:56.631101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f49d1-7470-4adf-9acc-8f40cb012e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:56.638936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f49d1-7470-4adf-9acc-8f40cb012e2b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:56.688188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 904/1819] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:26:56.690753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f49d1-7470-4adf-9acc-8f40cb012e2b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:56.712882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f49d1-7470-4adf-9acc-8f40cb012e2b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:56.718873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fff25c58-9448-4779-ae21-11047ea59868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64d6f42597d4ab591577dff4c287703'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.721482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc19e6-42f9-4799-adf5-01d67439da5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:56.722780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc19e6-42f9-4799-adf5-01d67439da5a admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:56.723279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc19e6-42f9-4799-adf5-01d67439da5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:56.723763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc19e6-42f9-4799-adf5-01d67439da5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:56.725011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc19e6-42f9-4799-adf5-01d67439da5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:56.728976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc19e6-42f9-4799-adf5-01d67439da5a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:56.746339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:56.748211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fff25c58-9448-4779-ae21-11047ea59868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64d6f42597d4ab591577dff4c287703'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.752220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fff25c58-9448-4779-ae21-11047ea59868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64d6f42597d4ab591577dff4c287703'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.768161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fff25c58-9448-4779-ae21-11047ea59868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64d6f42597d4ab591577dff4c287703'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.793755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fff25c58-9448-4779-ae21-11047ea59868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64d6f42597d4ab591577dff4c287703'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:56.803237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 905/1820] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:26:56 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:26:56.805751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc19e6-42f9-4799-adf5-01d67439da5a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:56.814627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-04fc19e6-42f9-4799-adf5-01d67439da5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:56.824901 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 916/1821] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:56 2022] DELETE /identity/v3/users/c64d6f42597d4ab591577dff4c287703 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:26:56.828593 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af5017-901e-47f0-900f-12d292f25206 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:56.829086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af5017-901e-47f0-900f-12d292f25206 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:56.829539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14af5017-901e-47f0-900f-12d292f25206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:56.865652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-14af5017-901e-47f0-900f-12d292f25206 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:56.868425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-14af5017-901e-47f0-900f-12d292f25206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:57.019554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc19e6-42f9-4799-adf5-01d67439da5a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:57.113728 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 917/1822] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:57.115346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af5017-901e-47f0-900f-12d292f25206 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:57.116943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-14af5017-901e-47f0-900f-12d292f25206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:57.141395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14af5017-901e-47f0-900f-12d292f25206 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:57.147795 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:57.148688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:57.148971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:57.149250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] PATH_INFO: `/v3/projects/a31d4d7270a942fa84698d04d807d59b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:57.152020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a31d4d7270a942fa84698d04d807d59b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:57.155061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:57.201939 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb33d22-0663-4aed-8d60-d36b36888c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:57.203745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb33d22-0663-4aed-8d60-d36b36888c25 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:57.204223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb33d22-0663-4aed-8d60-d36b36888c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:57.204802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb33d22-0663-4aed-8d60-d36b36888c25 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:57.207330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb33d22-0663-4aed-8d60-d36b36888c25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:57.212979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb33d22-0663-4aed-8d60-d36b36888c25 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:57.249224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:57.264609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a31d4d7270a942fa84698d04d807d59b', 'request_id': 'req-fc525ff4-842a-4542-8a58-d8b8985eb38d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:57.271047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 906/1823] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:26:56 2022] POST /identity/v3/users => generated 286 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:57.272843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb33d22-0663-4aed-8d60-d36b36888c25 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:57.277131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-edb33d22-0663-4aed-8d60-d36b36888c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:57.283784 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a31d4d7270a942fa84698d04d807d59b', 'request_id': 'req-fc525ff4-842a-4542-8a58-d8b8985eb38d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:57.304039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a31d4d7270a942fa84698d04d807d59b', 'request_id': 'req-fc525ff4-842a-4542-8a58-d8b8985eb38d'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:57.358655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] The token cache is being invalidate because project a31d4d7270a942fa84698d04d807d59b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:57.359798 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:57.370099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:57.373466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 918/1824] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:57 2022] DELETE /identity/v3/projects/a31d4d7270a942fa84698d04d807d59b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:57.375966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:57.378185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:57.434568 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb33d22-0663-4aed-8d60-d36b36888c25 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:57.539225 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:57.559270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc525ff4-842a-4542-8a58-d8b8985eb38d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:57.566992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c3bb5-c43d-4fe1-bed3-e2bc15fc1b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:57.567750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb023f-a81f-4f45-8d3f-650c8e2c5696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:57.568347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c3bb5-c43d-4fe1-bed3-e2bc15fc1b5f service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:57.568840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb023f-a81f-4f45-8d3f-650c8e2c5696 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:57.568999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c3bb5-c43d-4fe1-bed3-e2bc15fc1b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:57.569315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb023f-a81f-4f45-8d3f-650c8e2c5696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:57.569465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c3bb5-c43d-4fe1-bed3-e2bc15fc1b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:57.569776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb023f-a81f-4f45-8d3f-650c8e2c5696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:57.571293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb023f-a81f-4f45-8d3f-650c8e2c5696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:57.572773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-381c3bb5-c43d-4fe1-bed3-e2bc15fc1b5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:57.574482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb023f-a81f-4f45-8d3f-650c8e2c5696 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:57.595413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c3bb5-c43d-4fe1-bed3-e2bc15fc1b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:57.598030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 919/1825] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:57.598370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c3bb5-c43d-4fe1-bed3-e2bc15fc1b5f service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:57.599537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb023f-a81f-4f45-8d3f-650c8e2c5696 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:57.602535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ebdb023f-a81f-4f45-8d3f-650c8e2c5696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:57.617385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 907/1826] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:57.619032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c3bb5-c43d-4fe1-bed3-e2bc15fc1b5f service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:57.621936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb023f-a81f-4f45-8d3f-650c8e2c5696 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:57.629245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:57.630385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:57.630872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:57.631329 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] PATH_INFO: `/v3/users/7cc1bff8159448db9cc5ee6ab5302ffb` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:57.633565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc1bff8159448db9cc5ee6ab5302ffb)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:57.636769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:57.639838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c3bb5-c43d-4fe1-bed3-e2bc15fc1b5f service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:57.648601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8463d7c-d27f-471c-a5c5-fafc170a9f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:57.649922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8463d7c-d27f-471c-a5c5-fafc170a9f7e admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:57.650513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8463d7c-d27f-471c-a5c5-fafc170a9f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:57.651160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8463d7c-d27f-471c-a5c5-fafc170a9f7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:57.652698 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8463d7c-d27f-471c-a5c5-fafc170a9f7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:57.657382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8463d7c-d27f-471c-a5c5-fafc170a9f7e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:57.680691 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 908/1827] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:26:57.687795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8463d7c-d27f-471c-a5c5-fafc170a9f7e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:57.705822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8463d7c-d27f-471c-a5c5-fafc170a9f7e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:57.716505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff496b7-af0a-44b4-ba00-2bb5adfb4bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:57.718274 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff496b7-af0a-44b4-ba00-2bb5adfb4bdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:57.719343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff496b7-af0a-44b4-ba00-2bb5adfb4bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:57.719821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff496b7-af0a-44b4-ba00-2bb5adfb4bdb admin admin] PATH_INFO: `/v3/projects/d1a99ecabdd045a191a619668899ee75/users/bc4cd9e5c48641fab4a3ef7b182588f8/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:57.726846 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc1bff8159448db9cc5ee6ab5302ffb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:57.762657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff496b7-af0a-44b4-ba00-2bb5adfb4bdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a99ecabdd045a191a619668899ee75, user_id=bc4cd9e5c48641fab4a3ef7b182588f8, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:57.769262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff496b7-af0a-44b4-ba00-2bb5adfb4bdb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:57.786732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc1bff8159448db9cc5ee6ab5302ffb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:57.790293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc1bff8159448db9cc5ee6ab5302ffb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:57.802888 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 909/1828] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:26:57 2022] PUT /identity/v3/projects/d1a99ecabdd045a191a619668899ee75/users/bc4cd9e5c48641fab4a3ef7b182588f8/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:26:57.822996 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc1bff8159448db9cc5ee6ab5302ffb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:57.825104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ae36f-77e5-4f0d-a158-0b1e1397eb46 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:57.825620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ae36f-77e5-4f0d-a158-0b1e1397eb46 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:57.826124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ae36f-77e5-4f0d-a158-0b1e1397eb46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:57.840822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc1bff8159448db9cc5ee6ab5302ffb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:57.861335 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 920/1829] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:57 2022] DELETE /identity/v3/users/7cc1bff8159448db9cc5ee6ab5302ffb => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:57.902994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-567ae36f-77e5-4f0d-a158-0b1e1397eb46 None None] MFA Rules not processed for user `bc4cd9e5c48641fab4a3ef7b182588f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:57.905347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-567ae36f-77e5-4f0d-a158-0b1e1397eb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:58.104879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 910/1830] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:26:58.126469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:58.129330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:58.251736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:58.289480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c722d-e1b2-44df-8dfa-1f53286f4e51 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:58.299562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9223a-de51-4df2-bf08-930ba9cdfef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:58.300812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9223a-de51-4df2-bf08-930ba9cdfef5 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:58.301913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9223a-de51-4df2-bf08-930ba9cdfef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:58.302454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9223a-de51-4df2-bf08-930ba9cdfef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:58.347203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9223a-de51-4df2-bf08-930ba9cdfef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:58.354141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9223a-de51-4df2-bf08-930ba9cdfef5 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:58.386692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 921/1831] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:58.420727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ae36f-77e5-4f0d-a158-0b1e1397eb46 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:58.423678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-567ae36f-77e5-4f0d-a158-0b1e1397eb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:58.573434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ae36f-77e5-4f0d-a158-0b1e1397eb46 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:58.589109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825e12-8e81-4675-9082-9cd2b7395227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:58.590872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825e12-8e81-4675-9082-9cd2b7395227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:58.592167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825e12-8e81-4675-9082-9cd2b7395227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:58.592442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825e12-8e81-4675-9082-9cd2b7395227 admin admin] PATH_INFO: `/v3/users/bc4cd9e5c48641fab4a3ef7b182588f8` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:58.599498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb825e12-8e81-4675-9082-9cd2b7395227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc4cd9e5c48641fab4a3ef7b182588f8)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:58.599989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb825e12-8e81-4675-9082-9cd2b7395227 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:58.601207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9223a-de51-4df2-bf08-930ba9cdfef5 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:58.603422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6aa9223a-de51-4df2-bf08-930ba9cdfef5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:58.645302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9223a-de51-4df2-bf08-930ba9cdfef5 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:58.653915 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:58.655055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:58.655534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:58.655990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] PATH_INFO: `/v3/projects/ef83da28b788440491781b52fa2f498a` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:58.658963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef83da28b788440491781b52fa2f498a)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:58.661166 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:58.754466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:58.775446 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fb825e12-8e81-4675-9082-9cd2b7395227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc4cd9e5c48641fab4a3ef7b182588f8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:58.805030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fb825e12-8e81-4675-9082-9cd2b7395227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc4cd9e5c48641fab4a3ef7b182588f8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:58.828903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fb825e12-8e81-4675-9082-9cd2b7395227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc4cd9e5c48641fab4a3ef7b182588f8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:58.831343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef83da28b788440491781b52fa2f498a', 'request_id': 'req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:58.854007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef83da28b788440491781b52fa2f498a', 'request_id': 'req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:58.863454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fb825e12-8e81-4675-9082-9cd2b7395227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc4cd9e5c48641fab4a3ef7b182588f8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:58.864861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fb825e12-8e81-4675-9082-9cd2b7395227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc4cd9e5c48641fab4a3ef7b182588f8'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:58.872185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef83da28b788440491781b52fa2f498a', 'request_id': 'req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:58.884204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 911/1832] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:58 2022] DELETE /identity/v3/users/bc4cd9e5c48641fab4a3ef7b182588f8 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:26:58.892322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04644058-70cf-4c82-a00c-9d88db6d594d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:58.893424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04644058-70cf-4c82-a00c-9d88db6d594d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:58.893424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04644058-70cf-4c82-a00c-9d88db6d594d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:58.916911 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] The token cache is being invalidate because project ef83da28b788440491781b52fa2f498a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:58.917619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:58.919447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9ba154b2-c9ce-41f9-85d6-45d1adfa2160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:58.926349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 922/1833] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:58 2022] DELETE /identity/v3/projects/ef83da28b788440491781b52fa2f498a => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:26:58.940982 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499ed50-62a0-4ed8-bb57-450801ea229a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:58.941598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499ed50-62a0-4ed8-bb57-450801ea229a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:58.941839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-04644058-70cf-4c82-a00c-9d88db6d594d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:58.942258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499ed50-62a0-4ed8-bb57-450801ea229a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:58.951185 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-04644058-70cf-4c82-a00c-9d88db6d594d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:58.976650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8499ed50-62a0-4ed8-bb57-450801ea229a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:58.978254 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8499ed50-62a0-4ed8-bb57-450801ea229a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:59.120231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 912/1834] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:59.137867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 923/1835] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:59.138178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04644058-70cf-4c82-a00c-9d88db6d594d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:59.138178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-04644058-70cf-4c82-a00c-9d88db6d594d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:59.142384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86b86d-d1f2-4ab7-8e88-6da17f586a61 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:59.142923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86b86d-d1f2-4ab7-8e88-6da17f586a61 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:59.143434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86b86d-d1f2-4ab7-8e88-6da17f586a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:59.185939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ab86b86d-d1f2-4ab7-8e88-6da17f586a61 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:26:59.191075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ab86b86d-d1f2-4ab7-8e88-6da17f586a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:59.214403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 924/1836] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:26:59.218198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86b86d-d1f2-4ab7-8e88-6da17f586a61 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:59.253118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86b86d-d1f2-4ab7-8e88-6da17f586a61 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:59.279631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc62a6-e6c2-4455-ac3c-3ec9c4ef540f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:59.281700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc62a6-e6c2-4455-ac3c-3ec9c4ef540f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:59.282231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc62a6-e6c2-4455-ac3c-3ec9c4ef540f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:59.283352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc62a6-e6c2-4455-ac3c-3ec9c4ef540f admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:59.284527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc62a6-e6c2-4455-ac3c-3ec9c4ef540f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:26:59.285057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc62a6-e6c2-4455-ac3c-3ec9c4ef540f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:59.292764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc62a6-e6c2-4455-ac3c-3ec9c4ef540f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:59.297339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04644058-70cf-4c82-a00c-9d88db6d594d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:59.302728 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3d4f1-5def-4aab-865e-7202c2aa5ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:59.303836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3d4f1-5def-4aab-865e-7202c2aa5ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:59.304477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3d4f1-5def-4aab-865e-7202c2aa5ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:59.304850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d3d4f1-5def-4aab-865e-7202c2aa5ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:59.317733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3d4f1-5def-4aab-865e-7202c2aa5ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:59.319385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d3d4f1-5def-4aab-865e-7202c2aa5ad6 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:59.331727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 913/1837] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:26:59.357223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3d4f1-5def-4aab-865e-7202c2aa5ad6 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:59.363378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-26d3d4f1-5def-4aab-865e-7202c2aa5ad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:59.377489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 925/1838] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:26:59.380292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc62a6-e6c2-4455-ac3c-3ec9c4ef540f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:59.404288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d3d4f1-5def-4aab-865e-7202c2aa5ad6 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:59.416287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc62a6-e6c2-4455-ac3c-3ec9c4ef540f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:59.428858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:59.430945 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:59.432774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec1943-02e7-439b-a9ec-ac0c6ff5eed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:59.433089 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:59.433522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] PATH_INFO: `/v3/projects/9b236b3e528d4e2d842a7d5f5cab2f50` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:59.434311 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec1943-02e7-439b-a9ec-ac0c6ff5eed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:59.435604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec1943-02e7-439b-a9ec-ac0c6ff5eed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:59.436541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec1943-02e7-439b-a9ec-ac0c6ff5eed6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:59.437147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b236b3e528d4e2d842a7d5f5cab2f50)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:59.438771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec1943-02e7-439b-a9ec-ac0c6ff5eed6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:59.444069 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec1943-02e7-439b-a9ec-ac0c6ff5eed6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:59.446162 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:59.529245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 926/1839] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:26:59 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:59.539427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec1943-02e7-439b-a9ec-ac0c6ff5eed6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:59.602349 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b236b3e528d4e2d842a7d5f5cab2f50', 'request_id': 'req-dab1b889-bb8b-4788-9fad-f4e5e211093c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:59.602855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:59.624412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:26:59.636113 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b236b3e528d4e2d842a7d5f5cab2f50', 'request_id': 'req-dab1b889-bb8b-4788-9fad-f4e5e211093c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:59.653279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b236b3e528d4e2d842a7d5f5cab2f50', 'request_id': 'req-dab1b889-bb8b-4788-9fad-f4e5e211093c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:59.683496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] The token cache is being invalidate because project 9b236b3e528d4e2d842a7d5f5cab2f50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:26:59.683770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:59.684742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:26:59.690761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 914/1840] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:26:59 2022] DELETE /identity/v3/projects/9b236b3e528d4e2d842a7d5f5cab2f50 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:26:59.696465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:59.698251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:59.705871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec1943-02e7-439b-a9ec-ac0c6ff5eed6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:59.898889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1b889-bb8b-4788-9fad-f4e5e211093c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:26:59.910479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5e433-d8b8-4dea-a75e-b2c6a0957956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:59.910893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:26:59.911226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5e433-d8b8-4dea-a75e-b2c6a0957956 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:59.911517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5e433-d8b8-4dea-a75e-b2c6a0957956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:59.911799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5e433-d8b8-4dea-a75e-b2c6a0957956 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:59.912535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:26:59.913177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:26:59.913654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] PATH_INFO: `/v3/users/ae6f59fbc44840998271aa6286552b08` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:26:59.916113 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae6f59fbc44840998271aa6286552b08)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:59.917490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5e433-d8b8-4dea-a75e-b2c6a0957956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:26:59.919576 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5e433-d8b8-4dea-a75e-b2c6a0957956 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:59.922045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:26:59.951237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 927/1841] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:26:59 2022] POST /identity/v3/users => generated 289 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:26:59.957710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5e433-d8b8-4dea-a75e-b2c6a0957956 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:26:59.963650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9ca5e433-d8b8-4dea-a75e-b2c6a0957956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:26:59.996496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5e433-d8b8-4dea-a75e-b2c6a0957956 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:00.023291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae6f59fbc44840998271aa6286552b08'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:00.032781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae6f59fbc44840998271aa6286552b08'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:00.044885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecf5dc-3865-4498-803b-1adb309de35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:00.048155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecf5dc-3865-4498-803b-1adb309de35c admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:00.048643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecf5dc-3865-4498-803b-1adb309de35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:00.049036 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecf5dc-3865-4498-803b-1adb309de35c admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:00.050375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecf5dc-3865-4498-803b-1adb309de35c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:00.050757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae6f59fbc44840998271aa6286552b08'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:00.055646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecf5dc-3865-4498-803b-1adb309de35c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:00.083483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae6f59fbc44840998271aa6286552b08'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:00.084300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae6f59fbc44840998271aa6286552b08'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:00.094949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 915/1842] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:26:59 2022] DELETE /identity/v3/users/ae6f59fbc44840998271aa6286552b08 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:00.107048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 928/1843] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:00.108532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:00.110254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:00.251874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a5f328-0b55-4416-9ca1-1c79c0f50602 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:00.262353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3edfe5-4c12-4eae-aa94-9100de721300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:00.263613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3edfe5-4c12-4eae-aa94-9100de721300 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:00.263917 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3edfe5-4c12-4eae-aa94-9100de721300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:00.264204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3edfe5-4c12-4eae-aa94-9100de721300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:00.265296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3edfe5-4c12-4eae-aa94-9100de721300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:00.268068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3edfe5-4c12-4eae-aa94-9100de721300 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:00.275452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:00.298252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 916/1844] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:00.300405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecf5dc-3865-4498-803b-1adb309de35c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:00.323629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecf5dc-3865-4498-803b-1adb309de35c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:00.335830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94e5df-4ef1-44fa-8a46-9957af4bdbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:00.338266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94e5df-4ef1-44fa-8a46-9957af4bdbeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:00.339541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94e5df-4ef1-44fa-8a46-9957af4bdbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:00.340354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94e5df-4ef1-44fa-8a46-9957af4bdbeb admin admin] PATH_INFO: `/v3/projects/6aff8fb6a7904019a3a05ea4aa2cc420/users/ec1bcac8a3ff47b981c6923e74d658d2/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:00.346345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3edfe5-4c12-4eae-aa94-9100de721300 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:00.348556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8b3edfe5-4c12-4eae-aa94-9100de721300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:00.373194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3edfe5-4c12-4eae-aa94-9100de721300 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:00.378781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94e5df-4ef1-44fa-8a46-9957af4bdbeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aff8fb6a7904019a3a05ea4aa2cc420, user_id=ec1bcac8a3ff47b981c6923e74d658d2, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:00.379831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:00.380735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:00.381027 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:00.381304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] PATH_INFO: `/v3/projects/d1a99ecabdd045a191a619668899ee75` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:00.382946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a99ecabdd045a191a619668899ee75)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:00.385826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d94e5df-4ef1-44fa-8a46-9957af4bdbeb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:00.396406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:00.428074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 929/1845] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:00 2022] PUT /identity/v3/projects/6aff8fb6a7904019a3a05ea4aa2cc420/users/ec1bcac8a3ff47b981c6923e74d658d2/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:00.449762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595fabb-4159-4b5f-b6f0-a90b0aac5173 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:00.451399 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595fabb-4159-4b5f-b6f0-a90b0aac5173 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:00.451909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595fabb-4159-4b5f-b6f0-a90b0aac5173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:00.509201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a99ecabdd045a191a619668899ee75', 'request_id': 'req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:00.510165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5595fabb-4159-4b5f-b6f0-a90b0aac5173 None None] MFA Rules not processed for user `ec1bcac8a3ff47b981c6923e74d658d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:00.512719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5595fabb-4159-4b5f-b6f0-a90b0aac5173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:00.518148 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a99ecabdd045a191a619668899ee75', 'request_id': 'req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:00.530366 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a99ecabdd045a191a619668899ee75', 'request_id': 'req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:00.556129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] The token cache is being invalidate because project d1a99ecabdd045a191a619668899ee75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:27:00.557042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:00.558326 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4afca2ff-2134-44fd-b4b1-6e26b4ca3f58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:00.560301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 917/1846] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:27:00 2022] DELETE /identity/v3/projects/d1a99ecabdd045a191a619668899ee75 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:00.566949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fcb35-e2c4-4878-9820-9c12f2661564 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:00.567528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fcb35-e2c4-4878-9820-9c12f2661564 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:00.568698 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33fcb35-e2c4-4878-9820-9c12f2661564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:00.595740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a33fcb35-e2c4-4878-9820-9c12f2661564 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:00.597610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a33fcb35-e2c4-4878-9820-9c12f2661564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:00.688524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 930/1847] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:00.710304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595fabb-4159-4b5f-b6f0-a90b0aac5173 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:00.712956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5595fabb-4159-4b5f-b6f0-a90b0aac5173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:00.722439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 918/1848] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:00.731491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799ae44-ecf8-440e-bae8-cbc0fee164b0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:00.731794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799ae44-ecf8-440e-bae8-cbc0fee164b0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:00.732088 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799ae44-ecf8-440e-bae8-cbc0fee164b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:00.761850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:00.794127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3799ae44-ecf8-440e-bae8-cbc0fee164b0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:00.795527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3799ae44-ecf8-440e-bae8-cbc0fee164b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:00.820469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 919/1849] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:27:00.821476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799ae44-ecf8-440e-bae8-cbc0fee164b0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:00.834828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799ae44-ecf8-440e-bae8-cbc0fee164b0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:00.844410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572433a-675f-4c14-9873-c283256f06d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:00.845455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572433a-675f-4c14-9873-c283256f06d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:00.845827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572433a-675f-4c14-9873-c283256f06d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:00.846190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572433a-675f-4c14-9873-c283256f06d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:00.847051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572433a-675f-4c14-9873-c283256f06d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:27:00.847442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572433a-675f-4c14-9873-c283256f06d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:00.849612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572433a-675f-4c14-9873-c283256f06d0 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:00.861369 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595fabb-4159-4b5f-b6f0-a90b0aac5173 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:00.866686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2e592-ed18-4a3f-bf9a-80a2b4383580 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3778997232d44cd3a3c4fb3d53e4db53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:00.867692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2e592-ed18-4a3f-bf9a-80a2b4383580 service glance] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:00.868114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 920/1850] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:27:00.869279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572433a-675f-4c14-9873-c283256f06d0 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:00.871917 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5572433a-675f-4c14-9873-c283256f06d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:00.872126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2e592-ed18-4a3f-bf9a-80a2b4383580 service glance] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:00.872309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2e592-ed18-4a3f-bf9a-80a2b4383580 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:00.885086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2e592-ed18-4a3f-bf9a-80a2b4383580 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:00.887810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2e592-ed18-4a3f-bf9a-80a2b4383580 service glance] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:00.899581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 931/1851] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:27:00.900653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2e592-ed18-4a3f-bf9a-80a2b4383580 service glance] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:00.913724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2e592-ed18-4a3f-bf9a-80a2b4383580 service glance] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:00.919204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d7a10-cf48-49a6-91df-b3a0836feb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:00.919889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d7a10-cf48-49a6-91df-b3a0836feb44 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:00.920178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d7a10-cf48-49a6-91df-b3a0836feb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:00.920458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d7a10-cf48-49a6-91df-b3a0836feb44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:00.921206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d7a10-cf48-49a6-91df-b3a0836feb44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:00.922203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572433a-675f-4c14-9873-c283256f06d0 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:00.924296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d7a10-cf48-49a6-91df-b3a0836feb44 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:00.939614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:00.942001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:00.942518 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:00.943054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] PATH_INFO: `/v3/projects/3dc8ade0a5294f71ae2b924822c6fb91` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:00.959975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dc8ade0a5294f71ae2b924822c6fb91)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:00.962922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:00.996435 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 932/1852] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:00 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:01.002853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d7a10-cf48-49a6-91df-b3a0836feb44 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:01.080135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dc8ade0a5294f71ae2b924822c6fb91', 'request_id': 'req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:01.106447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:01.114340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dc8ade0a5294f71ae2b924822c6fb91', 'request_id': 'req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:01.125493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dc8ade0a5294f71ae2b924822c6fb91', 'request_id': 'req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:01.151943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d7a10-cf48-49a6-91df-b3a0836feb44 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:01.156488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] The token cache is being invalidate because project 3dc8ade0a5294f71ae2b924822c6fb91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:27:01.156878 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:01.157724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8dcb8362-0970-4db4-b6dd-479d5f9cf28e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:01.159898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 921/1853] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:27:00 2022] DELETE /identity/v3/projects/3dc8ade0a5294f71ae2b924822c6fb91 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:01.167237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6915f4c-7ee9-4c4f-b6e3-e8d53f222cbd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:01.167475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6915f4c-7ee9-4c4f-b6e3-e8d53f222cbd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:01.167774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6915f4c-7ee9-4c4f-b6e3-e8d53f222cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:01.204551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f6915f4c-7ee9-4c4f-b6e3-e8d53f222cbd None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:01.207533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f6915f4c-7ee9-4c4f-b6e3-e8d53f222cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:01.305179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2718f7-2ba0-4b7f-aee9-a709fe5dd425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:01.306099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2718f7-2ba0-4b7f-aee9-a709fe5dd425 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:01.306292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2718f7-2ba0-4b7f-aee9-a709fe5dd425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:01.306479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2718f7-2ba0-4b7f-aee9-a709fe5dd425 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:01.307277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2718f7-2ba0-4b7f-aee9-a709fe5dd425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:01.317729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2718f7-2ba0-4b7f-aee9-a709fe5dd425 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:01.352696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 922/1854] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:01.358198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 933/1855] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:00 2022] POST /identity/v3/users => generated 289 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:27:01.362341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6915f4c-7ee9-4c4f-b6e3-e8d53f222cbd None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:01.363664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bf37e-02e5-4a0b-894f-626b0aab6d68 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:01.364086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bf37e-02e5-4a0b-894f-626b0aab6d68 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:01.364376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bf37e-02e5-4a0b-894f-626b0aab6d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:01.364775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f6915f4c-7ee9-4c4f-b6e3-e8d53f222cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:01.391390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-195bf37e-02e5-4a0b-894f-626b0aab6d68 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:01.392287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6915f4c-7ee9-4c4f-b6e3-e8d53f222cbd None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:01.392959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-195bf37e-02e5-4a0b-894f-626b0aab6d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:01.398932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226b51a-bf66-49a4-9183-ebc5992e48d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:01.400098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226b51a-bf66-49a4-9183-ebc5992e48d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:01.400578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226b51a-bf66-49a4-9183-ebc5992e48d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:01.401238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226b51a-bf66-49a4-9183-ebc5992e48d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:01.402199 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226b51a-bf66-49a4-9183-ebc5992e48d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:01.404255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226b51a-bf66-49a4-9183-ebc5992e48d6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:01.416715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 934/1856] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:27:01.418419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bf37e-02e5-4a0b-894f-626b0aab6d68 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:01.434877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 923/1857] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:01.435078 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bf37e-02e5-4a0b-894f-626b0aab6d68 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:01.437034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226b51a-bf66-49a4-9183-ebc5992e48d6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:01.449357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d8acc-590a-4b1a-9859-6820df460597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:01.453188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226b51a-bf66-49a4-9183-ebc5992e48d6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:01.453636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d8acc-590a-4b1a-9859-6820df460597 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:01.454220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d8acc-590a-4b1a-9859-6820df460597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:01.455415 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d8acc-590a-4b1a-9859-6820df460597 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:01.456455 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d8acc-590a-4b1a-9859-6820df460597 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:27:01.458824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d8acc-590a-4b1a-9859-6820df460597 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:01.463671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85777855-bf28-4dd6-817a-aa8bf9767776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:01.464936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85777855-bf28-4dd6-817a-aa8bf9767776 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:01.465125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d8acc-590a-4b1a-9859-6820df460597 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:01.465695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85777855-bf28-4dd6-817a-aa8bf9767776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:01.466313 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85777855-bf28-4dd6-817a-aa8bf9767776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:01.467854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85777855-bf28-4dd6-817a-aa8bf9767776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:01.471200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:01.473621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85777855-bf28-4dd6-817a-aa8bf9767776 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:01.497832 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 924/1858] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:01.506375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85777855-bf28-4dd6-817a-aa8bf9767776 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:01.510570 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 935/1859] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:27:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:27:01.511815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d8acc-590a-4b1a-9859-6820df460597 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:01.525110 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d8acc-590a-4b1a-9859-6820df460597 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:01.532922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa42ce-5de5-4edb-987f-06da457f74d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:01.533905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa42ce-5de5-4edb-987f-06da457f74d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:01.534237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa42ce-5de5-4edb-987f-06da457f74d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:01.534479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfa42ce-5de5-4edb-987f-06da457f74d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:01.535234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa42ce-5de5-4edb-987f-06da457f74d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:01.536815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85777855-bf28-4dd6-817a-aa8bf9767776 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:01.537232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfa42ce-5de5-4edb-987f-06da457f74d9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:01.548609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed43a44-8298-43cc-8a16-e50fc67deda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:01.549675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed43a44-8298-43cc-8a16-e50fc67deda1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:01.550415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed43a44-8298-43cc-8a16-e50fc67deda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:01.550947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed43a44-8298-43cc-8a16-e50fc67deda1 admin admin] PATH_INFO: `/v3/projects/7d2810dd314046fdbf9e10dbd78ec6d7/users/67c4217ad8cc48f8a241c031f011a997/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:01.577911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed43a44-8298-43cc-8a16-e50fc67deda1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d2810dd314046fdbf9e10dbd78ec6d7, user_id=67c4217ad8cc48f8a241c031f011a997, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:01.579470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 936/1860] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:01 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:01.584050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa42ce-5de5-4edb-987f-06da457f74d9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:01.585041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed43a44-8298-43cc-8a16-e50fc67deda1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:01.605400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 925/1861] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:01 2022] PUT /identity/v3/projects/7d2810dd314046fdbf9e10dbd78ec6d7/users/67c4217ad8cc48f8a241c031f011a997/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:01.617258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a593a-7583-46ae-84cd-90f2fa3253cc None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:01.617767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a593a-7583-46ae-84cd-90f2fa3253cc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:01.618255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a593a-7583-46ae-84cd-90f2fa3253cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:01.673590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a00a593a-7583-46ae-84cd-90f2fa3253cc None None] MFA Rules not processed for user `67c4217ad8cc48f8a241c031f011a997`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:01.675436 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a00a593a-7583-46ae-84cd-90f2fa3253cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:01.699182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfa42ce-5de5-4edb-987f-06da457f74d9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:01.843661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d302dab-52d1-4f5c-9889-66e938ea65ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:01.844879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d302dab-52d1-4f5c-9889-66e938ea65ed admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:01.845245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d302dab-52d1-4f5c-9889-66e938ea65ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:01.845797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d302dab-52d1-4f5c-9889-66e938ea65ed admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:01.846847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d302dab-52d1-4f5c-9889-66e938ea65ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:01.849526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d302dab-52d1-4f5c-9889-66e938ea65ed admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:01.859899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 926/1862] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:01.877687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a593a-7583-46ae-84cd-90f2fa3253cc None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:01.879744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a00a593a-7583-46ae-84cd-90f2fa3253cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:01.894083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 937/1863] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:01 2022] POST /identity/v3/users => generated 283 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:01.899993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d302dab-52d1-4f5c-9889-66e938ea65ed admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:01.918267 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d302dab-52d1-4f5c-9889-66e938ea65ed admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:01.939779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f4224-9d72-4cfc-a7db-50990eb0cde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:01.940376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f4224-9d72-4cfc-a7db-50990eb0cde0 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:01.940704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f4224-9d72-4cfc-a7db-50990eb0cde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:01.941158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f4224-9d72-4cfc-a7db-50990eb0cde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:01.941936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f4224-9d72-4cfc-a7db-50990eb0cde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:01.945575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f4224-9d72-4cfc-a7db-50990eb0cde0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:01.948409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:01.977840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 938/1864] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:27:01.982479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f4224-9d72-4cfc-a7db-50990eb0cde0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:02.020800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f4224-9d72-4cfc-a7db-50990eb0cde0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:02.036265 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8dbbd-a74d-450f-baed-f25f7a9d9c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:02.038707 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a593a-7583-46ae-84cd-90f2fa3253cc None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:02.039822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8dbbd-a74d-450f-baed-f25f7a9d9c42 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:02.040288 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8dbbd-a74d-450f-baed-f25f7a9d9c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:02.040604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8dbbd-a74d-450f-baed-f25f7a9d9c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:02.041563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8dbbd-a74d-450f-baed-f25f7a9d9c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:02.043375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8dbbd-a74d-450f-baed-f25f7a9d9c42 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:02.051629 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae1dd82-0e24-469f-9569-85dd8cc861fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:02.052295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1dd82-0e24-469f-9569-85dd8cc861fb service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:02.052669 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1dd82-0e24-469f-9569-85dd8cc861fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:02.052951 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae1dd82-0e24-469f-9569-85dd8cc861fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:02.065841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1dd82-0e24-469f-9569-85dd8cc861fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:02.067801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae1dd82-0e24-469f-9569-85dd8cc861fb service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:02.074974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 939/1865] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:02.082117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8dbbd-a74d-450f-baed-f25f7a9d9c42 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:02.106206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 927/1866] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:27:02.112186 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8dbbd-a74d-450f-baed-f25f7a9d9c42 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:02.122170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf9d6b-a1ae-4054-a22f-5af7004645c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:02.123980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf9d6b-a1ae-4054-a22f-5af7004645c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:02.124632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf9d6b-a1ae-4054-a22f-5af7004645c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:02.126034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf9d6b-a1ae-4054-a22f-5af7004645c3 admin admin] PATH_INFO: `/v3/projects/833a6c995cae47ccb58997ce172b2916/users/9cf3c371996c4bf78c769f84c16ac360/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:02.159862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf9d6b-a1ae-4054-a22f-5af7004645c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=833a6c995cae47ccb58997ce172b2916, user_id=9cf3c371996c4bf78c769f84c16ac360, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:02.163312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf9d6b-a1ae-4054-a22f-5af7004645c3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:02.182742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 940/1867] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:02 2022] PUT /identity/v3/projects/833a6c995cae47ccb58997ce172b2916/users/9cf3c371996c4bf78c769f84c16ac360/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:02.195972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40947f-edee-4567-8d38-8b9ef5b86a59 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:02.196315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40947f-edee-4567-8d38-8b9ef5b86a59 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:02.196477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40947f-edee-4567-8d38-8b9ef5b86a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:02.242816 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cc40947f-edee-4567-8d38-8b9ef5b86a59 None None] MFA Rules not processed for user `9cf3c371996c4bf78c769f84c16ac360`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:02.245290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cc40947f-edee-4567-8d38-8b9ef5b86a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:02.433540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 928/1868] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:27:02.438935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf9d6b-a1ae-4054-a22f-5af7004645c3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:02.475398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:02.574023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf9d6b-a1ae-4054-a22f-5af7004645c3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:02.605058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad09ede-640d-4429-9771-ddbd18dafab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:02.607458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad09ede-640d-4429-9771-ddbd18dafab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:02.608511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad09ede-640d-4429-9771-ddbd18dafab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:02.610480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad09ede-640d-4429-9771-ddbd18dafab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:02.612645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad09ede-640d-4429-9771-ddbd18dafab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:02.616910 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad09ede-640d-4429-9771-ddbd18dafab3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:02.713133 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 941/1869] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:02 2022] POST /identity/v3/projects => generated 365 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:27:02.713746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40947f-edee-4567-8d38-8b9ef5b86a59 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:02.732112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc40947f-edee-4567-8d38-8b9ef5b86a59 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:02.876566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb9ba8-3f78-4d04-ae9c-4b7fc871aa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:02.877775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb9ba8-3f78-4d04-ae9c-4b7fc871aa73 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:02.878304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb9ba8-3f78-4d04-ae9c-4b7fc871aa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:02.878837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb9ba8-3f78-4d04-ae9c-4b7fc871aa73 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:02.880071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb9ba8-3f78-4d04-ae9c-4b7fc871aa73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:02.883243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb9ba8-3f78-4d04-ae9c-4b7fc871aa73 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:02.915630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 929/1870] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:02 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:02.920573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad09ede-640d-4429-9771-ddbd18dafab3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:02.934698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad09ede-640d-4429-9771-ddbd18dafab3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:02.942021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf035fba-0430-429e-a8e3-2e05ab85d544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:02.942749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf035fba-0430-429e-a8e3-2e05ab85d544 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:02.943045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf035fba-0430-429e-a8e3-2e05ab85d544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:02.943322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf035fba-0430-429e-a8e3-2e05ab85d544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:02.944196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf035fba-0430-429e-a8e3-2e05ab85d544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:02.946032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf035fba-0430-429e-a8e3-2e05ab85d544 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:02.968636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 942/1871] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:27:02.969373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb9ba8-3f78-4d04-ae9c-4b7fc871aa73 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:02.993483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb9ba8-3f78-4d04-ae9c-4b7fc871aa73 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:03.013266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d706a14-d08b-4ce0-90d2-6ffd669138e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:03.014410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d706a14-d08b-4ce0-90d2-6ffd669138e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:03.014746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d706a14-d08b-4ce0-90d2-6ffd669138e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:03.015032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d706a14-d08b-4ce0-90d2-6ffd669138e4 admin admin] PATH_INFO: `/v3/projects/64e2631055c04db78c691b5795b6f86b/users/86c4284f88ac4dba840f69800906d8e4/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:03.055192 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d706a14-d08b-4ce0-90d2-6ffd669138e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e2631055c04db78c691b5795b6f86b, user_id=86c4284f88ac4dba840f69800906d8e4, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:03.057956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d706a14-d08b-4ce0-90d2-6ffd669138e4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:03.087589 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf035fba-0430-429e-a8e3-2e05ab85d544 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:03.088441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 930/1872] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:02 2022] PUT /identity/v3/projects/64e2631055c04db78c691b5795b6f86b/users/86c4284f88ac4dba840f69800906d8e4/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:03.103100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf035fba-0430-429e-a8e3-2e05ab85d544 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:03.213694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4dfb1-7428-4d4f-a0bf-43fcffec52a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:03.214706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4dfb1-7428-4d4f-a0bf-43fcffec52a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:03.215066 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4dfb1-7428-4d4f-a0bf-43fcffec52a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:03.216086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4dfb1-7428-4d4f-a0bf-43fcffec52a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:03.217202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4dfb1-7428-4d4f-a0bf-43fcffec52a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:03.219674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4dfb1-7428-4d4f-a0bf-43fcffec52a8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:03.240712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 943/1873] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:03.242207 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d706a14-d08b-4ce0-90d2-6ffd669138e4 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:03.258963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d706a14-d08b-4ce0-90d2-6ffd669138e4 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:03.274879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c5eaf-2ae0-47c4-bb5c-c507675d76f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:03.276696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c5eaf-2ae0-47c4-bb5c-c507675d76f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:03.277297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c5eaf-2ae0-47c4-bb5c-c507675d76f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:03.277926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c5eaf-2ae0-47c4-bb5c-c507675d76f8 admin admin] PATH_INFO: `/v3/projects/64e2631055c04db78c691b5795b6f86b/users/86c4284f88ac4dba840f69800906d8e4/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:03.282945 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c5eaf-2ae0-47c4-bb5c-c507675d76f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e2631055c04db78c691b5795b6f86b, user_id=86c4284f88ac4dba840f69800906d8e4, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:03.289053 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c5eaf-2ae0-47c4-bb5c-c507675d76f8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:03.315110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 931/1874] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:03 2022] PUT /identity/v3/projects/64e2631055c04db78c691b5795b6f86b/users/86c4284f88ac4dba840f69800906d8e4/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:03.335277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d9d2d-cead-4085-96ec-44781e19ff8e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:03.335884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d9d2d-cead-4085-96ec-44781e19ff8e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:03.336389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432d9d2d-cead-4085-96ec-44781e19ff8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:03.400599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-432d9d2d-cead-4085-96ec-44781e19ff8e None None] MFA Rules not processed for user `86c4284f88ac4dba840f69800906d8e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:03.403298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-432d9d2d-cead-4085-96ec-44781e19ff8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:03.615917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 944/1875] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:03.634095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c5eaf-2ae0-47c4-bb5c-c507675d76f8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:03.638131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9d6c5eaf-2ae0-47c4-bb5c-c507675d76f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:03.669461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:03.753710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c5eaf-2ae0-47c4-bb5c-c507675d76f8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:03.758561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d4ddf-5a68-4d1a-a48c-1fb79fe154b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:03.759555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4ddf-5a68-4d1a-a48c-1fb79fe154b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:03.759964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4ddf-5a68-4d1a-a48c-1fb79fe154b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:03.760427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4ddf-5a68-4d1a-a48c-1fb79fe154b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:03.778543 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d4ddf-5a68-4d1a-a48c-1fb79fe154b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:03.781646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d4ddf-5a68-4d1a-a48c-1fb79fe154b5 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:03.808061 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 932/1876] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:27:04.026110 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d9d2d-cead-4085-96ec-44781e19ff8e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:04.046334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432d9d2d-cead-4085-96ec-44781e19ff8e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:04.054204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a85b7e-84af-4ff7-9cde-b34c88d4a0e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:04.055454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a85b7e-84af-4ff7-9cde-b34c88d4a0e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:04.056013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a85b7e-84af-4ff7-9cde-b34c88d4a0e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:04.056553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a85b7e-84af-4ff7-9cde-b34c88d4a0e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:04.116541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:04.297776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a85b7e-84af-4ff7-9cde-b34c88d4a0e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:04.301583 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a85b7e-84af-4ff7-9cde-b34c88d4a0e1 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:04.341285 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 945/1877] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:27:08.725884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d4ddf-5a68-4d1a-a48c-1fb79fe154b5 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:08.831327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d4ddf-5a68-4d1a-a48c-1fb79fe154b5 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:08.840227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:08.841145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:08.841434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:08.841717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] PATH_INFO: `/v3/users/9cf3c371996c4bf78c769f84c16ac360` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:08.845364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cf3c371996c4bf78c769f84c16ac360)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:08.847582 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:08.954795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cf3c371996c4bf78c769f84c16ac360'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:08.965198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cf3c371996c4bf78c769f84c16ac360'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:08.979737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cf3c371996c4bf78c769f84c16ac360'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:08.997330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cf3c371996c4bf78c769f84c16ac360'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:08.998470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cf3c371996c4bf78c769f84c16ac360'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:09.008731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 933/1878] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:27:08 2022] DELETE /identity/v3/users/9cf3c371996c4bf78c769f84c16ac360 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:09.014912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bf7e3-732c-430d-ae98-002f53063877 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:09.016067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bf7e3-732c-430d-ae98-002f53063877 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:09.016528 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bf7e3-732c-430d-ae98-002f53063877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:09.040518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-cf6bf7e3-732c-430d-ae98-002f53063877 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:09.041873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cf6bf7e3-732c-430d-ae98-002f53063877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:09.115765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 946/1879] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:09.130719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:09.133056 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:09.193422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:09.217018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a5dfa-2798-4c19-9c61-370203c4be7b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:09.221866 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944b474-a149-4809-b738-7a14eae08ccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:09.222552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944b474-a149-4809-b738-7a14eae08ccd service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:09.223042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944b474-a149-4809-b738-7a14eae08ccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:09.223328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9944b474-a149-4809-b738-7a14eae08ccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:09.237492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944b474-a149-4809-b738-7a14eae08ccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:09.240855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9944b474-a149-4809-b738-7a14eae08ccd service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:09.264359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 934/1880] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:27:09.782520 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bf7e3-732c-430d-ae98-002f53063877 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:09.785230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cf6bf7e3-732c-430d-ae98-002f53063877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:09.811498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bf7e3-732c-430d-ae98-002f53063877 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:09.819941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:09.822014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:09.822228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:09.822880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] PATH_INFO: `/v3/projects/833a6c995cae47ccb58997ce172b2916` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:09.825375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=833a6c995cae47ccb58997ce172b2916)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:09.829716 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:09.975639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '833a6c995cae47ccb58997ce172b2916', 'request_id': 'req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:09.991099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '833a6c995cae47ccb58997ce172b2916', 'request_id': 'req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:10.012949 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '833a6c995cae47ccb58997ce172b2916', 'request_id': 'req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:10.036078 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] The token cache is being invalidate because project 833a6c995cae47ccb58997ce172b2916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:27:10.036752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:10.037985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:10.045147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 947/1881] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:27:09 2022] DELETE /identity/v3/projects/833a6c995cae47ccb58997ce172b2916 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:10.047408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944b474-a149-4809-b738-7a14eae08ccd service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:10.066992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9944b474-a149-4809-b738-7a14eae08ccd service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:10.173413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de75263-7127-4933-b830-98d3fd255d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:10.174359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de75263-7127-4933-b830-98d3fd255d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:10.174645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de75263-7127-4933-b830-98d3fd255d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:10.174935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de75263-7127-4933-b830-98d3fd255d7f admin admin] PATH_INFO: `/v3/users/86c4284f88ac4dba840f69800906d8e4` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:10.176678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de75263-7127-4933-b830-98d3fd255d7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=86c4284f88ac4dba840f69800906d8e4)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:10.178946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de75263-7127-4933-b830-98d3fd255d7f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:10.247211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9de75263-7127-4933-b830-98d3fd255d7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86c4284f88ac4dba840f69800906d8e4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:10.262457 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9de75263-7127-4933-b830-98d3fd255d7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86c4284f88ac4dba840f69800906d8e4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:10.275265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9de75263-7127-4933-b830-98d3fd255d7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86c4284f88ac4dba840f69800906d8e4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:10.302952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9de75263-7127-4933-b830-98d3fd255d7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86c4284f88ac4dba840f69800906d8e4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:10.305706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-9de75263-7127-4933-b830-98d3fd255d7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86c4284f88ac4dba840f69800906d8e4'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:10.317396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 935/1882] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:27:10 2022] DELETE /identity/v3/users/86c4284f88ac4dba840f69800906d8e4 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:10.771138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:10.774306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:10.800269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadac2e0-e9c9-4c8a-b2b0-e12e638d9e63 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:10.857853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:10.884064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:10.884960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:10.885243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:10.885520 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] PATH_INFO: `/v3/projects/64e2631055c04db78c691b5795b6f86b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:10.886900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e2631055c04db78c691b5795b6f86b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:10.888938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:10.965444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e2631055c04db78c691b5795b6f86b', 'request_id': 'req-6555ebfd-e295-4400-a411-67665ceacad6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:10.980330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e2631055c04db78c691b5795b6f86b', 'request_id': 'req-6555ebfd-e295-4400-a411-67665ceacad6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:10.989555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e2631055c04db78c691b5795b6f86b', 'request_id': 'req-6555ebfd-e295-4400-a411-67665ceacad6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:11.012853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] The token cache is being invalidate because project 64e2631055c04db78c691b5795b6f86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:27:11.013599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:11.022645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6555ebfd-e295-4400-a411-67665ceacad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:11.030647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 948/1883] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:27:10 2022] DELETE /identity/v3/projects/64e2631055c04db78c691b5795b6f86b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:11.038407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e71bd-7fc2-4d78-936e-06ad1e9b0b67 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:11.038780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e71bd-7fc2-4d78-936e-06ad1e9b0b67 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:11.039086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e71bd-7fc2-4d78-936e-06ad1e9b0b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:11.081355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-767e71bd-7fc2-4d78-936e-06ad1e9b0b67 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:11.083048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-767e71bd-7fc2-4d78-936e-06ad1e9b0b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:11.197802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:11.234515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 936/1884] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:27:11.248871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eb7b2-1672-49ab-a595-94a13f073e07 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:11.250281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eb7b2-1672-49ab-a595-94a13f073e07 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:11.251025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55eb7b2-1672-49ab-a595-94a13f073e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:11.294756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b55eb7b2-1672-49ab-a595-94a13f073e07 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:11.298034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b55eb7b2-1672-49ab-a595-94a13f073e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:11.321140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 949/1885] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:27:11.328760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e71bd-7fc2-4d78-936e-06ad1e9b0b67 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:11.353002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e71bd-7fc2-4d78-936e-06ad1e9b0b67 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:11.365814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa01d3f-fe7f-42da-a98a-c1ce8e77e017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:11.367295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01d3f-fe7f-42da-a98a-c1ce8e77e017 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:11.368123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01d3f-fe7f-42da-a98a-c1ce8e77e017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:11.368702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa01d3f-fe7f-42da-a98a-c1ce8e77e017 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:11.371405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa01d3f-fe7f-42da-a98a-c1ce8e77e017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:27:11.372152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa01d3f-fe7f-42da-a98a-c1ce8e77e017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:11.377405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa01d3f-fe7f-42da-a98a-c1ce8e77e017 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:11.412834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 937/1886] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:27:11.420081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55eb7b2-1672-49ab-a595-94a13f073e07 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:11.438896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55eb7b2-1672-49ab-a595-94a13f073e07 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:11.447974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e404f-6cf0-434c-820e-f740fbfa5f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:11.449396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e404f-6cf0-434c-820e-f740fbfa5f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:11.449796 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e404f-6cf0-434c-820e-f740fbfa5f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:11.450177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e404f-6cf0-434c-820e-f740fbfa5f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:11.451054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e404f-6cf0-434c-820e-f740fbfa5f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:11.453322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e404f-6cf0-434c-820e-f740fbfa5f62 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:11.492784 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 950/1887] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:11 2022] POST /identity/v3/projects => generated 379 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:27:11.498146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa01d3f-fe7f-42da-a98a-c1ce8e77e017 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:11.513148 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa01d3f-fe7f-42da-a98a-c1ce8e77e017 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:11.610995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c43c2-849e-4a0b-8c15-ff537acbb4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:11.611901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c43c2-849e-4a0b-8c15-ff537acbb4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:11.612260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c43c2-849e-4a0b-8c15-ff537acbb4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:11.612546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c43c2-849e-4a0b-8c15-ff537acbb4a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:11.613439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c43c2-849e-4a0b-8c15-ff537acbb4a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:11.615784 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c43c2-849e-4a0b-8c15-ff537acbb4a3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:11.651974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 938/1888] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:11 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:11.657097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e404f-6cf0-434c-820e-f740fbfa5f62 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:11.668364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e404f-6cf0-434c-820e-f740fbfa5f62 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:11.674063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19580ab-313e-4741-825d-58c6e85b21c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:11.674796 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19580ab-313e-4741-825d-58c6e85b21c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:11.675084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19580ab-313e-4741-825d-58c6e85b21c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:11.675369 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19580ab-313e-4741-825d-58c6e85b21c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:11.676142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19580ab-313e-4741-825d-58c6e85b21c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:11.677791 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19580ab-313e-4741-825d-58c6e85b21c5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:11.693998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 951/1889] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:11.697654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c43c2-849e-4a0b-8c15-ff537acbb4a3 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:11.713082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c43c2-849e-4a0b-8c15-ff537acbb4a3 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:11.722147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935d321-7f54-4326-94f4-191fd5156dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:11.723568 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935d321-7f54-4326-94f4-191fd5156dfc admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:11.724071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935d321-7f54-4326-94f4-191fd5156dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:11.724543 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935d321-7f54-4326-94f4-191fd5156dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:11.725891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935d321-7f54-4326-94f4-191fd5156dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:11.729118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4935d321-7f54-4326-94f4-191fd5156dfc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:11.753873 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 939/1890] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:11.754867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19580ab-313e-4741-825d-58c6e85b21c5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:11.773727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19580ab-313e-4741-825d-58c6e85b21c5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:11.781836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a221f6-fa0c-4ff7-ba72-0c8cb931d2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:11.783425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a221f6-fa0c-4ff7-ba72-0c8cb931d2c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:11.783938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a221f6-fa0c-4ff7-ba72-0c8cb931d2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:11.784477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a221f6-fa0c-4ff7-ba72-0c8cb931d2c8 admin admin] PATH_INFO: `/v3/projects/22d7ea88d26c4a10ab8b1134928b42fa/users/a599804c18824386b8d44a8bb798155f/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:11.811634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a221f6-fa0c-4ff7-ba72-0c8cb931d2c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d7ea88d26c4a10ab8b1134928b42fa, user_id=a599804c18824386b8d44a8bb798155f, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:11.816999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a221f6-fa0c-4ff7-ba72-0c8cb931d2c8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:11.842506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 952/1891] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:11 2022] PUT /identity/v3/projects/22d7ea88d26c4a10ab8b1134928b42fa/users/a599804c18824386b8d44a8bb798155f/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:27:11.857944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac66b-ef60-44a6-99b6-ba54d34400e0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:11.857944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac66b-ef60-44a6-99b6-ba54d34400e0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:11.857944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac66b-ef60-44a6-99b6-ba54d34400e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:11.889472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a68ac66b-ef60-44a6-99b6-ba54d34400e0 None None] MFA Rules not processed for user `a599804c18824386b8d44a8bb798155f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:11.891372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a68ac66b-ef60-44a6-99b6-ba54d34400e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:11.991146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 940/1892] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:11.996203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a221f6-fa0c-4ff7-ba72-0c8cb931d2c8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:12.086774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a221f6-fa0c-4ff7-ba72-0c8cb931d2c8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:12.091624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99044ce-3b4e-42ef-8917-321f19a6ff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:12.092460 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99044ce-3b4e-42ef-8917-321f19a6ff21 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:12.092819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99044ce-3b4e-42ef-8917-321f19a6ff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:12.093172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99044ce-3b4e-42ef-8917-321f19a6ff21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:12.094395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99044ce-3b4e-42ef-8917-321f19a6ff21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:12.096718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99044ce-3b4e-42ef-8917-321f19a6ff21 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:12.112282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:12.137742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 953/1893] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:11 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:27:12.139942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ac66b-ef60-44a6-99b6-ba54d34400e0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:12.164411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ac66b-ef60-44a6-99b6-ba54d34400e0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:12.285700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:12.303026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b4400-e121-41c3-93cd-98807ed17755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:12.304086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b4400-e121-41c3-93cd-98807ed17755 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:12.304585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b4400-e121-41c3-93cd-98807ed17755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:12.304934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074b4400-e121-41c3-93cd-98807ed17755 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:12.305916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b4400-e121-41c3-93cd-98807ed17755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:12.309199 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074b4400-e121-41c3-93cd-98807ed17755 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:12.340636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 941/1894] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:12 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:12.345736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99044ce-3b4e-42ef-8917-321f19a6ff21 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:12.362676 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99044ce-3b4e-42ef-8917-321f19a6ff21 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:12.369335 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42a04e-262b-4d8f-a2d3-9f066eb29dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:12.370213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42a04e-262b-4d8f-a2d3-9f066eb29dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:12.370707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42a04e-262b-4d8f-a2d3-9f066eb29dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:12.376643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42a04e-262b-4d8f-a2d3-9f066eb29dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:12.378149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42a04e-262b-4d8f-a2d3-9f066eb29dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:12.381780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42a04e-262b-4d8f-a2d3-9f066eb29dcd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:12.412480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 954/1895] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:27:12.416706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b4400-e121-41c3-93cd-98807ed17755 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:12.442654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074b4400-e121-41c3-93cd-98807ed17755 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:12.459262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b80f7e-d3bb-45fd-a2c6-31e20a7a1bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:12.461228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b80f7e-d3bb-45fd-a2c6-31e20a7a1bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:12.461761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b80f7e-d3bb-45fd-a2c6-31e20a7a1bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:12.462289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b80f7e-d3bb-45fd-a2c6-31e20a7a1bbc admin admin] PATH_INFO: `/v3/projects/d38f1668689545c6a1000a6fd4a75e8d/users/c1092312ebf64504a64c116ac28300a9/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:12.489713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b80f7e-d3bb-45fd-a2c6-31e20a7a1bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d38f1668689545c6a1000a6fd4a75e8d, user_id=c1092312ebf64504a64c116ac28300a9, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:12.493784 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b80f7e-d3bb-45fd-a2c6-31e20a7a1bbc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:12.509803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 942/1896] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:12 2022] PUT /identity/v3/projects/d38f1668689545c6a1000a6fd4a75e8d/users/c1092312ebf64504a64c116ac28300a9/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:12.516231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42a04e-262b-4d8f-a2d3-9f066eb29dcd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:12.539463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42a04e-262b-4d8f-a2d3-9f066eb29dcd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:12.639194 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c508e-9d41-46e3-8457-f25d12df0b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:12.640594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c508e-9d41-46e3-8457-f25d12df0b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:12.644325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c508e-9d41-46e3-8457-f25d12df0b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:12.644934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c508e-9d41-46e3-8457-f25d12df0b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:12.646339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c508e-9d41-46e3-8457-f25d12df0b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:12.649522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c508e-9d41-46e3-8457-f25d12df0b01 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:12.674975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 955/1897] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:12.677374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b80f7e-d3bb-45fd-a2c6-31e20a7a1bbc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:12.697837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b80f7e-d3bb-45fd-a2c6-31e20a7a1bbc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:12.707255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd4604-fd54-4e11-b9a4-22e448cb818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:12.708228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd4604-fd54-4e11-b9a4-22e448cb818d admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:12.708616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd4604-fd54-4e11-b9a4-22e448cb818d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:12.708896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd4604-fd54-4e11-b9a4-22e448cb818d admin admin] PATH_INFO: `/v3/projects/d38f1668689545c6a1000a6fd4a75e8d/users/c1092312ebf64504a64c116ac28300a9/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:12.711327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd4604-fd54-4e11-b9a4-22e448cb818d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d38f1668689545c6a1000a6fd4a75e8d, user_id=c1092312ebf64504a64c116ac28300a9, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:12.714096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd4604-fd54-4e11-b9a4-22e448cb818d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:12.743140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 943/1898] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:12 2022] PUT /identity/v3/projects/d38f1668689545c6a1000a6fd4a75e8d/users/c1092312ebf64504a64c116ac28300a9/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:12.745150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce7e8b-aa57-4ff0-b04c-6108fe4e13e5 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:12.745504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce7e8b-aa57-4ff0-b04c-6108fe4e13e5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:12.745931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce7e8b-aa57-4ff0-b04c-6108fe4e13e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:12.782865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8bce7e8b-aa57-4ff0-b04c-6108fe4e13e5 None None] MFA Rules not processed for user `c1092312ebf64504a64c116ac28300a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:12.784217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8bce7e8b-aa57-4ff0-b04c-6108fe4e13e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:12.870184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 956/1899] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:12.882606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd4604-fd54-4e11-b9a4-22e448cb818d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:12.884456 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3ddd4604-fd54-4e11-b9a4-22e448cb818d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:12.964692 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd4604-fd54-4e11-b9a4-22e448cb818d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:12.970624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7026b54-7077-420d-a138-7de5cfcf4d2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:12.971333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7026b54-7077-420d-a138-7de5cfcf4d2f service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:12.971621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7026b54-7077-420d-a138-7de5cfcf4d2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:12.971905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7026b54-7077-420d-a138-7de5cfcf4d2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:12.984675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7026b54-7077-420d-a138-7de5cfcf4d2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:12.986749 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7026b54-7077-420d-a138-7de5cfcf4d2f service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:13.004714 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 944/1900] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:27:13.148051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce7e8b-aa57-4ff0-b04c-6108fe4e13e5 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:13.164468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce7e8b-aa57-4ff0-b04c-6108fe4e13e5 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:13.169726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c68f93-2c72-4e00-85de-390dcb15245c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:13.170549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c68f93-2c72-4e00-85de-390dcb15245c service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:13.171049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c68f93-2c72-4e00-85de-390dcb15245c service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:13.171495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c68f93-2c72-4e00-85de-390dcb15245c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:13.273217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c68f93-2c72-4e00-85de-390dcb15245c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:13.276829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c68f93-2c72-4e00-85de-390dcb15245c service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:13.288282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:13.298804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 957/1901] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:27:22.756025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7026b54-7077-420d-a138-7de5cfcf4d2f service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:22.850676 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7026b54-7077-420d-a138-7de5cfcf4d2f service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:22.856472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:22.857886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:22.858454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:22.858841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] PATH_INFO: `/v3/users/a599804c18824386b8d44a8bb798155f` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:22.861134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a599804c18824386b8d44a8bb798155f)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:22.874636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:22.956727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a599804c18824386b8d44a8bb798155f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:22.961408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:22.972359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a599804c18824386b8d44a8bb798155f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:23.002245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a599804c18824386b8d44a8bb798155f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:23.051628 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a599804c18824386b8d44a8bb798155f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:23.053880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a599804c18824386b8d44a8bb798155f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:23.086243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 945/1902] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:27:22 2022] DELETE /identity/v3/users/a599804c18824386b8d44a8bb798155f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:23.090551 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f167a-a4f2-4c6f-98e8-3837131d7c03 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:23.091332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f167a-a4f2-4c6f-98e8-3837131d7c03 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:23.091332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f167a-a4f2-4c6f-98e8-3837131d7c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:23.128785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ff8f167a-a4f2-4c6f-98e8-3837131d7c03 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:23.130060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ff8f167a-a4f2-4c6f-98e8-3837131d7c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:23.207497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 958/1903] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:23.221670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:23.223572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:23.298621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5c7d2-48ea-4566-96c8-e0f9bcad3a0e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:23.302670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ec8b7-5621-486d-be9c-f98eaf3b45f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:23.303290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ec8b7-5621-486d-be9c-f98eaf3b45f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:23.303581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ec8b7-5621-486d-be9c-f98eaf3b45f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:23.303862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ec8b7-5621-486d-be9c-f98eaf3b45f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:23.316812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ec8b7-5621-486d-be9c-f98eaf3b45f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:23.318317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ec8b7-5621-486d-be9c-f98eaf3b45f1 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:23.329845 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 946/1904] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:27:23.716906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f167a-a4f2-4c6f-98e8-3837131d7c03 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:23.719200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ff8f167a-a4f2-4c6f-98e8-3837131d7c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:23.742047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f167a-a4f2-4c6f-98e8-3837131d7c03 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:23.748939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:23.749934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:23.750279 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:23.750598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] PATH_INFO: `/v3/projects/22d7ea88d26c4a10ab8b1134928b42fa` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:23.752276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d7ea88d26c4a10ab8b1134928b42fa)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:23.754886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:23.840856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d7ea88d26c4a10ab8b1134928b42fa', 'request_id': 'req-2467ab01-b0de-4a49-911f-d090ff0c718e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:23.853031 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d7ea88d26c4a10ab8b1134928b42fa', 'request_id': 'req-2467ab01-b0de-4a49-911f-d090ff0c718e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:23.864755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d7ea88d26c4a10ab8b1134928b42fa', 'request_id': 'req-2467ab01-b0de-4a49-911f-d090ff0c718e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:23.882668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] The token cache is being invalidate because project 22d7ea88d26c4a10ab8b1134928b42fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:27:23.882968 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:23.883733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:23.885667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 959/1905] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:27:23 2022] DELETE /identity/v3/projects/22d7ea88d26c4a10ab8b1134928b42fa => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:23.891435 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ec8b7-5621-486d-be9c-f98eaf3b45f1 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:23.902801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ec8b7-5621-486d-be9c-f98eaf3b45f1 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:23.983299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa628857-10e9-4cc0-9547-45b4202fb039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:23.984426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa628857-10e9-4cc0-9547-45b4202fb039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:23.984911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa628857-10e9-4cc0-9547-45b4202fb039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:23.985399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa628857-10e9-4cc0-9547-45b4202fb039 admin admin] PATH_INFO: `/v3/users/c1092312ebf64504a64c116ac28300a9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:23.986837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa628857-10e9-4cc0-9547-45b4202fb039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1092312ebf64504a64c116ac28300a9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:23.988906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa628857-10e9-4cc0-9547-45b4202fb039 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:24.056973 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aa628857-10e9-4cc0-9547-45b4202fb039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1092312ebf64504a64c116ac28300a9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:24.065520 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aa628857-10e9-4cc0-9547-45b4202fb039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1092312ebf64504a64c116ac28300a9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:24.073805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aa628857-10e9-4cc0-9547-45b4202fb039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1092312ebf64504a64c116ac28300a9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:24.086217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:24.104210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aa628857-10e9-4cc0-9547-45b4202fb039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1092312ebf64504a64c116ac28300a9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:24.105436 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aa628857-10e9-4cc0-9547-45b4202fb039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1092312ebf64504a64c116ac28300a9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:24.116837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 947/1906] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:27:23 2022] DELETE /identity/v3/users/c1092312ebf64504a64c116ac28300a9 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:24.735103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:24.737244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:24.752484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467ab01-b0de-4a49-911f-d090ff0c718e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:24.833889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:24.835595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:24.836242 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:24.836868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] PATH_INFO: `/v3/projects/d38f1668689545c6a1000a6fd4a75e8d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:24.839474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d38f1668689545c6a1000a6fd4a75e8d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:24.843916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:24.889657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:24.917746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd38f1668689545c6a1000a6fd4a75e8d', 'request_id': 'req-c8f922bf-c7ad-4076-997e-51da026b85a6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:24.930462 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd38f1668689545c6a1000a6fd4a75e8d', 'request_id': 'req-c8f922bf-c7ad-4076-997e-51da026b85a6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:24.943510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd38f1668689545c6a1000a6fd4a75e8d', 'request_id': 'req-c8f922bf-c7ad-4076-997e-51da026b85a6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:24.962730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] The token cache is being invalidate because project d38f1668689545c6a1000a6fd4a75e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:27:24.963348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:24.964338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-c8f922bf-c7ad-4076-997e-51da026b85a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:24.966755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 960/1907] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:27:24 2022] DELETE /identity/v3/projects/d38f1668689545c6a1000a6fd4a75e8d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:27:24.975631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b7622-b54a-4f4b-ab6f-e0ff1c2ad55d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:24.976246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b7622-b54a-4f4b-ab6f-e0ff1c2ad55d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:24.981076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b7622-b54a-4f4b-ab6f-e0ff1c2ad55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:25.040655 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9f3b7622-b54a-4f4b-ab6f-e0ff1c2ad55d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:25.042519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9f3b7622-b54a-4f4b-ab6f-e0ff1c2ad55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:25.132759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 948/1908] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:25.141829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c504a08-67c2-438e-8627-7579b44b0852 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:25.142394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c504a08-67c2-438e-8627-7579b44b0852 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:25.142970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c504a08-67c2-438e-8627-7579b44b0852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:25.174459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4c504a08-67c2-438e-8627-7579b44b0852 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:25.176177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4c504a08-67c2-438e-8627-7579b44b0852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:25.193136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 961/1909] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:27:25.198787 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b7622-b54a-4f4b-ab6f-e0ff1c2ad55d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:25.213551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b7622-b54a-4f4b-ab6f-e0ff1c2ad55d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:25.219893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe62d3-b929-412b-a113-79949ab9a2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:25.220675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe62d3-b929-412b-a113-79949ab9a2db admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:25.221034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe62d3-b929-412b-a113-79949ab9a2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:25.221381 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe62d3-b929-412b-a113-79949ab9a2db admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:25.222206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe62d3-b929-412b-a113-79949ab9a2db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:27:25.222609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe62d3-b929-412b-a113-79949ab9a2db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:25.224387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe62d3-b929-412b-a113-79949ab9a2db admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:25.240452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 949/1910] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:27:25.245072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c504a08-67c2-438e-8627-7579b44b0852 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:25.256467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c504a08-67c2-438e-8627-7579b44b0852 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:25.264083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5861c-a2c7-4075-92ea-c1d3ca20da72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:25.265721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5861c-a2c7-4075-92ea-c1d3ca20da72 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:25.266081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5861c-a2c7-4075-92ea-c1d3ca20da72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:25.266426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5861c-a2c7-4075-92ea-c1d3ca20da72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:25.267237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5861c-a2c7-4075-92ea-c1d3ca20da72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:25.269287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5861c-a2c7-4075-92ea-c1d3ca20da72 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:25.305646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 962/1911] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:25 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:27:25.311158 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe62d3-b929-412b-a113-79949ab9a2db admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:25.323624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe62d3-b929-412b-a113-79949ab9a2db admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:25.418018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb68246-f517-4466-8cb1-80cd4a7e8672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:25.418860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb68246-f517-4466-8cb1-80cd4a7e8672 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:25.419216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb68246-f517-4466-8cb1-80cd4a7e8672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:25.419561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb68246-f517-4466-8cb1-80cd4a7e8672 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:25.420359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb68246-f517-4466-8cb1-80cd4a7e8672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:25.423172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb68246-f517-4466-8cb1-80cd4a7e8672 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:25.447945 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 950/1912] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:25 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:27:25.453122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5861c-a2c7-4075-92ea-c1d3ca20da72 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:25.469159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5861c-a2c7-4075-92ea-c1d3ca20da72 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:25.475638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510248e1-9098-4d9b-af6f-390c35d3496c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:25.476374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510248e1-9098-4d9b-af6f-390c35d3496c admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:25.476661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510248e1-9098-4d9b-af6f-390c35d3496c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:25.476942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510248e1-9098-4d9b-af6f-390c35d3496c admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:25.477706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510248e1-9098-4d9b-af6f-390c35d3496c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:25.479498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510248e1-9098-4d9b-af6f-390c35d3496c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:25.492306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 963/1913] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:27:25.497311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb68246-f517-4466-8cb1-80cd4a7e8672 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:25.517567 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb68246-f517-4466-8cb1-80cd4a7e8672 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:25.525636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41a481a-72fb-43da-a834-41ed439d0a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:25.526765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a481a-72fb-43da-a834-41ed439d0a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:25.527057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a481a-72fb-43da-a834-41ed439d0a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:25.527346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a481a-72fb-43da-a834-41ed439d0a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:25.528567 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a481a-72fb-43da-a834-41ed439d0a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:25.531814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41a481a-72fb-43da-a834-41ed439d0a1c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:25.556801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 951/1914] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:25.559042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510248e1-9098-4d9b-af6f-390c35d3496c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:25.581659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510248e1-9098-4d9b-af6f-390c35d3496c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:25.589710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf321fd-958d-4996-b4d5-0c997db54081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:25.591071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf321fd-958d-4996-b4d5-0c997db54081 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:25.591578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf321fd-958d-4996-b4d5-0c997db54081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:25.592100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf321fd-958d-4996-b4d5-0c997db54081 admin admin] PATH_INFO: `/v3/projects/02622d333fa24a0bbd1ef6f59276ba77/users/d88c5f05aab0458d8dc0a285bce33f28/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:25.619591 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf321fd-958d-4996-b4d5-0c997db54081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02622d333fa24a0bbd1ef6f59276ba77, user_id=d88c5f05aab0458d8dc0a285bce33f28, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:25.625721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf321fd-958d-4996-b4d5-0c997db54081 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:25.646948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 964/1915] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:25 2022] PUT /identity/v3/projects/02622d333fa24a0bbd1ef6f59276ba77/users/d88c5f05aab0458d8dc0a285bce33f28/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:25.652653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26cfce-0ef6-4e03-9c58-b8a5adbf37d1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:25.653165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26cfce-0ef6-4e03-9c58-b8a5adbf37d1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:25.653595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd26cfce-0ef6-4e03-9c58-b8a5adbf37d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:25.682149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cd26cfce-0ef6-4e03-9c58-b8a5adbf37d1 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:25.684387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cd26cfce-0ef6-4e03-9c58-b8a5adbf37d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:25.794220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:25.814171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 952/1916] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:27:25.827097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf321fd-958d-4996-b4d5-0c997db54081 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:25.829315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1cf321fd-958d-4996-b4d5-0c997db54081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:25.921373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf321fd-958d-4996-b4d5-0c997db54081 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:25.928243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d422e3-1828-43c3-9eb7-389e19071720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:25.929966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d422e3-1828-43c3-9eb7-389e19071720 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:25.929966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d422e3-1828-43c3-9eb7-389e19071720 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:25.929966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d422e3-1828-43c3-9eb7-389e19071720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:25.946835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d422e3-1828-43c3-9eb7-389e19071720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:25.947061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d422e3-1828-43c3-9eb7-389e19071720 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:25.962246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 965/1917] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:27:26.785307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c13114-1c7f-4029-943f-8a0bff92d33c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:26.785961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c13114-1c7f-4029-943f-8a0bff92d33c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:26.785961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c13114-1c7f-4029-943f-8a0bff92d33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:26.828183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-44c13114-1c7f-4029-943f-8a0bff92d33c None None] MFA Rules not processed for user `d88c5f05aab0458d8dc0a285bce33f28`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:26.829974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-44c13114-1c7f-4029-943f-8a0bff92d33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:26.931198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 953/1918] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:26.946753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d422e3-1828-43c3-9eb7-389e19071720 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:26.955707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e6d422e3-1828-43c3-9eb7-389e19071720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:27.042501 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d422e3-1828-43c3-9eb7-389e19071720 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:27.047191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79308f44-6329-4e6a-a2f9-5d8e0d6c6e4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:27.048165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79308f44-6329-4e6a-a2f9-5d8e0d6c6e4a service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:27.048538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79308f44-6329-4e6a-a2f9-5d8e0d6c6e4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:27.048892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79308f44-6329-4e6a-a2f9-5d8e0d6c6e4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:27.057399 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:27.066091 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79308f44-6329-4e6a-a2f9-5d8e0d6c6e4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:27.067839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79308f44-6329-4e6a-a2f9-5d8e0d6c6e4a service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:27.081873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 966/1919] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:27:33.782994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c8ee0-4b8f-491f-aed0-ef38b12f94d0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:33.783589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c8ee0-4b8f-491f-aed0-ef38b12f94d0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:33.783589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c8ee0-4b8f-491f-aed0-ef38b12f94d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:33.803580 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c32c8ee0-4b8f-491f-aed0-ef38b12f94d0 None None] MFA Rules not processed for user `d88c5f05aab0458d8dc0a285bce33f28`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:33.804818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c32c8ee0-4b8f-491f-aed0-ef38b12f94d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:33.812839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 954/1920] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:37.074689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:38.350119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79308f44-6329-4e6a-a2f9-5d8e0d6c6e4a service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:38.351655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-79308f44-6329-4e6a-a2f9-5d8e0d6c6e4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:38.366222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79308f44-6329-4e6a-a2f9-5d8e0d6c6e4a service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:38.371132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:38.372022 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:38.372304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:38.372584 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] PATH_INFO: `/v3/users/67c4217ad8cc48f8a241c031f011a997` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:38.373799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=67c4217ad8cc48f8a241c031f011a997)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:38.377109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:38.466611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67c4217ad8cc48f8a241c031f011a997'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:38.492384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67c4217ad8cc48f8a241c031f011a997'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:38.493355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67c4217ad8cc48f8a241c031f011a997'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:38.506450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67c4217ad8cc48f8a241c031f011a997'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:38.516206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67c4217ad8cc48f8a241c031f011a997'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:38.529463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 967/1921] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:27:38 2022] DELETE /identity/v3/users/67c4217ad8cc48f8a241c031f011a997 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:38.532674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443e6b6-8b67-4e7a-80ec-8ede47a57694 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:38.533278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443e6b6-8b67-4e7a-80ec-8ede47a57694 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:38.533278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443e6b6-8b67-4e7a-80ec-8ede47a57694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:38.558508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e443e6b6-8b67-4e7a-80ec-8ede47a57694 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:38.560035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e443e6b6-8b67-4e7a-80ec-8ede47a57694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:38.645604 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 955/1922] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:27:38.657856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:38.659675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:38.737738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decad901-f615-4929-acc7-c0a56badbb6c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:38.741708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d34c0-be15-4828-bc55-39bf30054174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:38.742687 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d34c0-be15-4828-bc55-39bf30054174 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:38.742855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d34c0-be15-4828-bc55-39bf30054174 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:38.743202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d34c0-be15-4828-bc55-39bf30054174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:38.754459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d34c0-be15-4828-bc55-39bf30054174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:38.756159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04d34c0-be15-4828-bc55-39bf30054174 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:38.768717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 968/1923] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:27:39.152296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443e6b6-8b67-4e7a-80ec-8ede47a57694 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:39.153972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e443e6b6-8b67-4e7a-80ec-8ede47a57694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:39.184693 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443e6b6-8b67-4e7a-80ec-8ede47a57694 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:39.192633 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:39.193295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:39.193582 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:39.194464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] PATH_INFO: `/v3/projects/7d2810dd314046fdbf9e10dbd78ec6d7` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:39.197407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d2810dd314046fdbf9e10dbd78ec6d7)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:39.199951 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:39.256861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d2810dd314046fdbf9e10dbd78ec6d7', 'request_id': 'req-e4736238-233b-4a87-a937-6d1e1c33d1db'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:39.262947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d2810dd314046fdbf9e10dbd78ec6d7', 'request_id': 'req-e4736238-233b-4a87-a937-6d1e1c33d1db'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:39.269461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d2810dd314046fdbf9e10dbd78ec6d7', 'request_id': 'req-e4736238-233b-4a87-a937-6d1e1c33d1db'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:39.283495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] The token cache is being invalidate because project 7d2810dd314046fdbf9e10dbd78ec6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:27:39.283794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:39.284466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e4736238-233b-4a87-a937-6d1e1c33d1db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:39.286074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 956/1924] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:27:39 2022] DELETE /identity/v3/projects/7d2810dd314046fdbf9e10dbd78ec6d7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:39.291365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d916ef-0997-4ec7-b654-0c1078dcdda4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:39.291714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d916ef-0997-4ec7-b654-0c1078dcdda4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:39.292050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d916ef-0997-4ec7-b654-0c1078dcdda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:39.311797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-25d916ef-0997-4ec7-b654-0c1078dcdda4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:39.314612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-25d916ef-0997-4ec7-b654-0c1078dcdda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:39.382153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 969/1925] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:39.390129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ff5d6-f3b3-4254-b687-e50bb0108787 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:39.390429 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ff5d6-f3b3-4254-b687-e50bb0108787 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:39.390740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ff5d6-f3b3-4254-b687-e50bb0108787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:39.397794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:39.412565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b86ff5d6-f3b3-4254-b687-e50bb0108787 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:39.413803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b86ff5d6-f3b3-4254-b687-e50bb0108787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:39.422461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 957/1926] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:39.427057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d916ef-0997-4ec7-b654-0c1078dcdda4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:39.440516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d916ef-0997-4ec7-b654-0c1078dcdda4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:39.446930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be68ee-4a95-4819-a943-47b55e085604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:39.447590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be68ee-4a95-4819-a943-47b55e085604 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:39.448468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be68ee-4a95-4819-a943-47b55e085604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:39.449157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be68ee-4a95-4819-a943-47b55e085604 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:39.451850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be68ee-4a95-4819-a943-47b55e085604 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:27:39.451850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be68ee-4a95-4819-a943-47b55e085604 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:39.452248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be68ee-4a95-4819-a943-47b55e085604 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:39.469801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 970/1927] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:27:39.471000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ff5d6-f3b3-4254-b687-e50bb0108787 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:39.480511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ff5d6-f3b3-4254-b687-e50bb0108787 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:39.489716 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0335-ba1d-4ede-bfeb-deaf81378454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:39.490416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0335-ba1d-4ede-bfeb-deaf81378454 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:39.490739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0335-ba1d-4ede-bfeb-deaf81378454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:39.491025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe0335-ba1d-4ede-bfeb-deaf81378454 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:39.491746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe0335-ba1d-4ede-bfeb-deaf81378454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:39.493717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe0335-ba1d-4ede-bfeb-deaf81378454 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:39.523630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 958/1928] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:39 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:39.528571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be68ee-4a95-4819-a943-47b55e085604 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:39.534162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:39.542707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be68ee-4a95-4819-a943-47b55e085604 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:39.633343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385536fa-e850-4cd6-a998-05e4e6ba82ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:39.633943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385536fa-e850-4cd6-a998-05e4e6ba82ac admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:39.634260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385536fa-e850-4cd6-a998-05e4e6ba82ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:39.634536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385536fa-e850-4cd6-a998-05e4e6ba82ac admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:39.635293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385536fa-e850-4cd6-a998-05e4e6ba82ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:39.637448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385536fa-e850-4cd6-a998-05e4e6ba82ac admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:39.667414 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 971/1929] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:39 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:27:39.668805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0335-ba1d-4ede-bfeb-deaf81378454 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:39.694821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe0335-ba1d-4ede-bfeb-deaf81378454 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:39.714546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0355f-4c82-448d-afcb-422f6cb5978d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:39.715547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0355f-4c82-448d-afcb-422f6cb5978d admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:39.716013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0355f-4c82-448d-afcb-422f6cb5978d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:39.716449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0355f-4c82-448d-afcb-422f6cb5978d admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:39.717373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0355f-4c82-448d-afcb-422f6cb5978d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:39.719899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe0355f-4c82-448d-afcb-422f6cb5978d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:39.743231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 959/1930] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:39.749312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385536fa-e850-4cd6-a998-05e4e6ba82ac admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:39.773738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385536fa-e850-4cd6-a998-05e4e6ba82ac admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:39.793325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5d6d8-1edf-4d30-a30b-2bc59a24c173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:39.794270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5d6d8-1edf-4d30-a30b-2bc59a24c173 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:39.795076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5d6d8-1edf-4d30-a30b-2bc59a24c173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:39.795395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5d6d8-1edf-4d30-a30b-2bc59a24c173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:39.797231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5d6d8-1edf-4d30-a30b-2bc59a24c173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:39.799887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5d6d8-1edf-4d30-a30b-2bc59a24c173 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:39.822327 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 972/1931] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:39.823701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0355f-4c82-448d-afcb-422f6cb5978d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:39.839549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe0355f-4c82-448d-afcb-422f6cb5978d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:39.848502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbb01a-035f-47d0-a25a-3dc8e2711549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:39.849660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbb01a-035f-47d0-a25a-3dc8e2711549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:39.850049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbb01a-035f-47d0-a25a-3dc8e2711549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:39.850530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbb01a-035f-47d0-a25a-3dc8e2711549 admin admin] PATH_INFO: `/v3/projects/071d4d7bb2bf4f5eab792a8de77d8b85/users/2e2289b569ae4943a22e42b3aaf9d10a/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:39.869862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbb01a-035f-47d0-a25a-3dc8e2711549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=071d4d7bb2bf4f5eab792a8de77d8b85, user_id=2e2289b569ae4943a22e42b3aaf9d10a, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:39.873047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbb01a-035f-47d0-a25a-3dc8e2711549 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:39.894522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 960/1932] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:39 2022] PUT /identity/v3/projects/071d4d7bb2bf4f5eab792a8de77d8b85/users/2e2289b569ae4943a22e42b3aaf9d10a/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:39.905602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55415555-04fb-43ae-929d-e977e792ae12 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:39.906441 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55415555-04fb-43ae-929d-e977e792ae12 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:39.906771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55415555-04fb-43ae-929d-e977e792ae12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:39.941975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-55415555-04fb-43ae-929d-e977e792ae12 None None] MFA Rules not processed for user `2e2289b569ae4943a22e42b3aaf9d10a`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:39.943438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-55415555-04fb-43ae-929d-e977e792ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:40.020808 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 973/1933] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:40.039204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbb01a-035f-47d0-a25a-3dc8e2711549 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:40.040979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0ddbb01a-035f-47d0-a25a-3dc8e2711549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:40.141644 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbb01a-035f-47d0-a25a-3dc8e2711549 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:40.146130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a4bbd-c8fa-4f1c-9f49-daf894740605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:40.147008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a4bbd-c8fa-4f1c-9f49-daf894740605 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:40.147352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a4bbd-c8fa-4f1c-9f49-daf894740605 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:40.147597 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a4bbd-c8fa-4f1c-9f49-daf894740605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:40.159758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a4bbd-c8fa-4f1c-9f49-daf894740605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:40.161367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a4bbd-c8fa-4f1c-9f49-daf894740605 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:40.178094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 961/1934] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:27:44.642698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55415555-04fb-43ae-929d-e977e792ae12 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:44.779911 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55415555-04fb-43ae-929d-e977e792ae12 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:44.791898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:44.792885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:44.793176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:44.793652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] PATH_INFO: `/v3/users/2e2289b569ae4943a22e42b3aaf9d10a` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:44.795245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2289b569ae4943a22e42b3aaf9d10a)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:44.798025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:44.816706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:44.890053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2289b569ae4943a22e42b3aaf9d10a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:44.908555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2289b569ae4943a22e42b3aaf9d10a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:44.911601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2289b569ae4943a22e42b3aaf9d10a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:44.932167 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2289b569ae4943a22e42b3aaf9d10a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:44.944664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2289b569ae4943a22e42b3aaf9d10a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:44.952407 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 974/1935] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:27:44 2022] DELETE /identity/v3/users/2e2289b569ae4943a22e42b3aaf9d10a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:44.957998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee174c51-9cd2-4a74-8f66-29000c2d5d75 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:44.958771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee174c51-9cd2-4a74-8f66-29000c2d5d75 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:44.959227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee174c51-9cd2-4a74-8f66-29000c2d5d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:44.984521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ee174c51-9cd2-4a74-8f66-29000c2d5d75 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:44.985709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ee174c51-9cd2-4a74-8f66-29000c2d5d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:45.054021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 962/1936] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:45.064025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:45.065565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:45.127461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e5e066-013b-4bcc-82c4-dffde78f0237 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:45.131087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d820a60-a849-4382-b615-5c9d5f19c312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:45.131744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d820a60-a849-4382-b615-5c9d5f19c312 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:45.132027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d820a60-a849-4382-b615-5c9d5f19c312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:45.132301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d820a60-a849-4382-b615-5c9d5f19c312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:45.142308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d820a60-a849-4382-b615-5c9d5f19c312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:45.143827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d820a60-a849-4382-b615-5c9d5f19c312 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:45.154484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 975/1937] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:27:45.509307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee174c51-9cd2-4a74-8f66-29000c2d5d75 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:45.510914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ee174c51-9cd2-4a74-8f66-29000c2d5d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:45.525942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee174c51-9cd2-4a74-8f66-29000c2d5d75 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:45.531108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:45.532276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:45.532684 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:45.533069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] PATH_INFO: `/v3/projects/071d4d7bb2bf4f5eab792a8de77d8b85` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:45.535165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=071d4d7bb2bf4f5eab792a8de77d8b85)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:45.537183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:45.588004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:45.599979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '071d4d7bb2bf4f5eab792a8de77d8b85', 'request_id': 'req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:45.606280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '071d4d7bb2bf4f5eab792a8de77d8b85', 'request_id': 'req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:45.617114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '071d4d7bb2bf4f5eab792a8de77d8b85', 'request_id': 'req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:45.636129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] The token cache is being invalidate because project 071d4d7bb2bf4f5eab792a8de77d8b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:27:45.636129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:45.636444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6f5bd79c-96ef-40f3-b96b-a0ee2e682c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:45.638342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 963/1938] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:27:45 2022] DELETE /identity/v3/projects/071d4d7bb2bf4f5eab792a8de77d8b85 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:27:45.645289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de54cc-e6a9-44fd-8712-64f49ea3bbf8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:45.645980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de54cc-e6a9-44fd-8712-64f49ea3bbf8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:45.645980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de54cc-e6a9-44fd-8712-64f49ea3bbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:45.664656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e2de54cc-e6a9-44fd-8712-64f49ea3bbf8 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:45.665877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e2de54cc-e6a9-44fd-8712-64f49ea3bbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:45.742862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 976/1939] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:45.745133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9dc1c-0f0d-4d09-9029-80fa1bfee1f4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:45.745453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9dc1c-0f0d-4d09-9029-80fa1bfee1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:45.745736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9dc1c-0f0d-4d09-9029-80fa1bfee1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:45.767244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-bdd9dc1c-0f0d-4d09-9029-80fa1bfee1f4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:45.768387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bdd9dc1c-0f0d-4d09-9029-80fa1bfee1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:45.777641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 964/1940] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:27:45.782367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de54cc-e6a9-44fd-8712-64f49ea3bbf8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:45.793174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de54cc-e6a9-44fd-8712-64f49ea3bbf8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:45.798962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3becd6-af06-4e2d-97f7-20accd4b31cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:45.800060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3becd6-af06-4e2d-97f7-20accd4b31cb admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:45.800060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3becd6-af06-4e2d-97f7-20accd4b31cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:45.800361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3becd6-af06-4e2d-97f7-20accd4b31cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:45.801094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3becd6-af06-4e2d-97f7-20accd4b31cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:27:45.801401 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3becd6-af06-4e2d-97f7-20accd4b31cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:45.803090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3becd6-af06-4e2d-97f7-20accd4b31cb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:45.818670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 977/1941] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:27:45.822818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9dc1c-0f0d-4d09-9029-80fa1bfee1f4 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:45.834105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9dc1c-0f0d-4d09-9029-80fa1bfee1f4 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:45.839245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f13962-0141-4d58-a406-4a48176cf89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:45.839979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f13962-0141-4d58-a406-4a48176cf89a admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:45.840262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f13962-0141-4d58-a406-4a48176cf89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:45.840544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f13962-0141-4d58-a406-4a48176cf89a admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:45.841658 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f13962-0141-4d58-a406-4a48176cf89a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:45.843760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f13962-0141-4d58-a406-4a48176cf89a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:45.871858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 965/1942] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:45 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:45.877129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3becd6-af06-4e2d-97f7-20accd4b31cb admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:45.889802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3becd6-af06-4e2d-97f7-20accd4b31cb admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:45.916138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:45.965872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc6679-9d90-4270-be84-56b3f5497f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:45.966553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc6679-9d90-4270-be84-56b3f5497f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:45.966873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc6679-9d90-4270-be84-56b3f5497f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:45.967154 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc6679-9d90-4270-be84-56b3f5497f69 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:45.967854 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc6679-9d90-4270-be84-56b3f5497f69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:45.969854 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc6679-9d90-4270-be84-56b3f5497f69 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:45.995764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 978/1943] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:45 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:45.997127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f13962-0141-4d58-a406-4a48176cf89a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:46.013066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f13962-0141-4d58-a406-4a48176cf89a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:46.022480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7f254-2217-407f-b42e-07fe9bb52382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:46.023123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7f254-2217-407f-b42e-07fe9bb52382 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:46.023578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7f254-2217-407f-b42e-07fe9bb52382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:46.024032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7f254-2217-407f-b42e-07fe9bb52382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:46.024930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7f254-2217-407f-b42e-07fe9bb52382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:46.027367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7f254-2217-407f-b42e-07fe9bb52382 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:46.043519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 966/1944] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:46.044279 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc6679-9d90-4270-be84-56b3f5497f69 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:46.060104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc6679-9d90-4270-be84-56b3f5497f69 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:46.069470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653ff1b9-561f-4e78-b0ce-cda9e89f4113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:46.075838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ff1b9-561f-4e78-b0ce-cda9e89f4113 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:46.076887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ff1b9-561f-4e78-b0ce-cda9e89f4113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:46.077472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653ff1b9-561f-4e78-b0ce-cda9e89f4113 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:46.080384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ff1b9-561f-4e78-b0ce-cda9e89f4113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:46.083618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653ff1b9-561f-4e78-b0ce-cda9e89f4113 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:46.097532 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 979/1945] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:27:46.102260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7f254-2217-407f-b42e-07fe9bb52382 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:46.132278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7f254-2217-407f-b42e-07fe9bb52382 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:46.138220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0462326-160a-442b-a2c6-10c2ec1d954a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:46.139837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0462326-160a-442b-a2c6-10c2ec1d954a admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:46.140563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0462326-160a-442b-a2c6-10c2ec1d954a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:46.141227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0462326-160a-442b-a2c6-10c2ec1d954a admin admin] PATH_INFO: `/v3/projects/f50eb3e07f4746ca84d4610e2d339dc8/users/956adb80f4ae4d2ea759afab00fccd68/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:46.159221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0462326-160a-442b-a2c6-10c2ec1d954a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50eb3e07f4746ca84d4610e2d339dc8, user_id=956adb80f4ae4d2ea759afab00fccd68, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:46.162630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0462326-160a-442b-a2c6-10c2ec1d954a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:46.180881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 967/1946] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:46 2022] PUT /identity/v3/projects/f50eb3e07f4746ca84d4610e2d339dc8/users/956adb80f4ae4d2ea759afab00fccd68/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:46.194747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d439a0e-2db2-402a-b4e9-473e48a75d9b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:46.195028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d439a0e-2db2-402a-b4e9-473e48a75d9b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:46.195317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d439a0e-2db2-402a-b4e9-473e48a75d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:46.231420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-1d439a0e-2db2-402a-b4e9-473e48a75d9b None None] MFA Rules not processed for user `956adb80f4ae4d2ea759afab00fccd68`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:46.233700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1d439a0e-2db2-402a-b4e9-473e48a75d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:46.316837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 980/1947] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:27:46.321303 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0462326-160a-442b-a2c6-10c2ec1d954a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:46.387532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0462326-160a-442b-a2c6-10c2ec1d954a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:46.393659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1593c-54bd-42a6-928f-3d55e134f917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:46.394452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1593c-54bd-42a6-928f-3d55e134f917 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:46.394889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1593c-54bd-42a6-928f-3d55e134f917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:46.395257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1593c-54bd-42a6-928f-3d55e134f917 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:46.396156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1593c-54bd-42a6-928f-3d55e134f917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:46.398470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1593c-54bd-42a6-928f-3d55e134f917 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:46.430520 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 968/1948] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:46 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:46.431811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d439a0e-2db2-402a-b4e9-473e48a75d9b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:46.444172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d439a0e-2db2-402a-b4e9-473e48a75d9b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:46.508351 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681faa9d-43d7-43a3-a5a2-f7ba4ae5548d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:46.509163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681faa9d-43d7-43a3-a5a2-f7ba4ae5548d admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:46.509453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681faa9d-43d7-43a3-a5a2-f7ba4ae5548d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:46.509909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681faa9d-43d7-43a3-a5a2-f7ba4ae5548d admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:46.510428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681faa9d-43d7-43a3-a5a2-f7ba4ae5548d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:46.512425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681faa9d-43d7-43a3-a5a2-f7ba4ae5548d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:46.540164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 981/1949] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:46 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:46.548782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1593c-54bd-42a6-928f-3d55e134f917 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:46.564168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1593c-54bd-42a6-928f-3d55e134f917 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:46.571594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b85ad-5ae8-42e2-8a01-418c634ba792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:46.572291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b85ad-5ae8-42e2-8a01-418c634ba792 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:46.572581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b85ad-5ae8-42e2-8a01-418c634ba792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:46.572858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b85ad-5ae8-42e2-8a01-418c634ba792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:46.573605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b85ad-5ae8-42e2-8a01-418c634ba792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:46.575335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b85ad-5ae8-42e2-8a01-418c634ba792 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:46.594001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 969/1950] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:46.596207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681faa9d-43d7-43a3-a5a2-f7ba4ae5548d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:46.609070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681faa9d-43d7-43a3-a5a2-f7ba4ae5548d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:46.614339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30362b4a-3e3c-44de-8ac6-f8e33afcf156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:46.615345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30362b4a-3e3c-44de-8ac6-f8e33afcf156 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:46.615629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30362b4a-3e3c-44de-8ac6-f8e33afcf156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:46.615909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30362b4a-3e3c-44de-8ac6-f8e33afcf156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:46.616654 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30362b4a-3e3c-44de-8ac6-f8e33afcf156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:46.618389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30362b4a-3e3c-44de-8ac6-f8e33afcf156 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:46.630630 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 982/1951] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:46.634999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b85ad-5ae8-42e2-8a01-418c634ba792 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:46.646362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b85ad-5ae8-42e2-8a01-418c634ba792 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:46.651229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d2d7a-b326-4591-9cf4-ca91b4773b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:46.652163 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d2d7a-b326-4591-9cf4-ca91b4773b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:46.652417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d2d7a-b326-4591-9cf4-ca91b4773b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:46.652698 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d2d7a-b326-4591-9cf4-ca91b4773b05 admin admin] PATH_INFO: `/v3/projects/ce2942b959b641d1bc0dde5437f9a094/users/059c35b81a6f4388b215e53c6e24a2b6/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:46.669213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d2d7a-b326-4591-9cf4-ca91b4773b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2942b959b641d1bc0dde5437f9a094, user_id=059c35b81a6f4388b215e53c6e24a2b6, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:46.672221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d2d7a-b326-4591-9cf4-ca91b4773b05 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:46.690142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 970/1952] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:46 2022] PUT /identity/v3/projects/ce2942b959b641d1bc0dde5437f9a094/users/059c35b81a6f4388b215e53c6e24a2b6/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:46.705125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531134d-a0e1-4971-94b2-0804d0251a7e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:46.705964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531134d-a0e1-4971-94b2-0804d0251a7e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:46.705964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a531134d-a0e1-4971-94b2-0804d0251a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:46.721347 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:46.741412 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a531134d-a0e1-4971-94b2-0804d0251a7e None None] MFA Rules not processed for user `059c35b81a6f4388b215e53c6e24a2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:46.743071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a531134d-a0e1-4971-94b2-0804d0251a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:46.821948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 983/1953] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:46.826675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d2d7a-b326-4591-9cf4-ca91b4773b05 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:46.845500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:46.900317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d2d7a-b326-4591-9cf4-ca91b4773b05 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:46.904946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203d779-2cf5-49d4-9251-e182d3c5c643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:46.905615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203d779-2cf5-49d4-9251-e182d3c5c643 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:46.906086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203d779-2cf5-49d4-9251-e182d3c5c643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:46.906392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203d779-2cf5-49d4-9251-e182d3c5c643 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:46.907126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203d779-2cf5-49d4-9251-e182d3c5c643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:46.909067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e203d779-2cf5-49d4-9251-e182d3c5c643 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:46.940811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 971/1954] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:46 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:46.942090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531134d-a0e1-4971-94b2-0804d0251a7e None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:46.953653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a531134d-a0e1-4971-94b2-0804d0251a7e None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:47.018027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd24cf-ae32-4131-8944-af0fb660bb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:47.018752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd24cf-ae32-4131-8944-af0fb660bb03 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:47.019036 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd24cf-ae32-4131-8944-af0fb660bb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:47.019309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbd24cf-ae32-4131-8944-af0fb660bb03 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:47.020003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd24cf-ae32-4131-8944-af0fb660bb03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:47.021947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbd24cf-ae32-4131-8944-af0fb660bb03 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:47.043481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 984/1955] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:46 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:47.048879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203d779-2cf5-49d4-9251-e182d3c5c643 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:47.060102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203d779-2cf5-49d4-9251-e182d3c5c643 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:47.065672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aeba03-5252-4972-b259-b7f5fab86d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:47.066343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeba03-5252-4972-b259-b7f5fab86d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:47.066676 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeba03-5252-4972-b259-b7f5fab86d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:47.066963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeba03-5252-4972-b259-b7f5fab86d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:47.067697 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aeba03-5252-4972-b259-b7f5fab86d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:47.069303 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aeba03-5252-4972-b259-b7f5fab86d68 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:47.080343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 972/1956] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:47.084719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd24cf-ae32-4131-8944-af0fb660bb03 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:47.095719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbd24cf-ae32-4131-8944-af0fb660bb03 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:47.101152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b789a2-02c1-4c64-a15d-3ea9affd0459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:47.102194 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b789a2-02c1-4c64-a15d-3ea9affd0459 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:47.102539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b789a2-02c1-4c64-a15d-3ea9affd0459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:47.102979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b789a2-02c1-4c64-a15d-3ea9affd0459 admin admin] PATH_INFO: `/v3/projects/236af605fe7a4609a6d31f6654b6b346/users/dadd52dcf4a4489eb0c0fea5b80ba772/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:47.121131 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b789a2-02c1-4c64-a15d-3ea9affd0459 admin admin] RBAC: Authorizing `identity:create_grant(project_id=236af605fe7a4609a6d31f6654b6b346, user_id=dadd52dcf4a4489eb0c0fea5b80ba772, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:47.124245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b789a2-02c1-4c64-a15d-3ea9affd0459 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:47.141073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 985/1957] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:47 2022] PUT /identity/v3/projects/236af605fe7a4609a6d31f6654b6b346/users/dadd52dcf4a4489eb0c0fea5b80ba772/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:47.145206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aeba03-5252-4972-b259-b7f5fab86d68 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:47.155466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aeba03-5252-4972-b259-b7f5fab86d68 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:47.215252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39fc2c-5e84-452e-98c1-e995ad0f05f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:47.215981 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39fc2c-5e84-452e-98c1-e995ad0f05f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:47.216273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39fc2c-5e84-452e-98c1-e995ad0f05f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:47.216555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39fc2c-5e84-452e-98c1-e995ad0f05f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:47.217295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39fc2c-5e84-452e-98c1-e995ad0f05f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:47.219658 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f39fc2c-5e84-452e-98c1-e995ad0f05f2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:47.233458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 973/1958] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:47.235504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b789a2-02c1-4c64-a15d-3ea9affd0459 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:47.246688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b789a2-02c1-4c64-a15d-3ea9affd0459 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:47.251363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b759b29-9356-451a-a0d4-3ded597aee74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:47.252523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b759b29-9356-451a-a0d4-3ded597aee74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:47.252809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b759b29-9356-451a-a0d4-3ded597aee74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:47.253085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b759b29-9356-451a-a0d4-3ded597aee74 admin admin] PATH_INFO: `/v3/projects/236af605fe7a4609a6d31f6654b6b346/users/dadd52dcf4a4489eb0c0fea5b80ba772/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:47.255062 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b759b29-9356-451a-a0d4-3ded597aee74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=236af605fe7a4609a6d31f6654b6b346, user_id=dadd52dcf4a4489eb0c0fea5b80ba772, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:47.257737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b759b29-9356-451a-a0d4-3ded597aee74 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:47.268571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 986/1959] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:47 2022] PUT /identity/v3/projects/236af605fe7a4609a6d31f6654b6b346/users/dadd52dcf4a4489eb0c0fea5b80ba772/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:27:47.280055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8a9d6-679e-4d24-8c94-390faa43c342 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:47.280575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8a9d6-679e-4d24-8c94-390faa43c342 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:47.281594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8a9d6-679e-4d24-8c94-390faa43c342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:47.325944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-87d8a9d6-679e-4d24-8c94-390faa43c342 None None] MFA Rules not processed for user `dadd52dcf4a4489eb0c0fea5b80ba772`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:47.327107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-87d8a9d6-679e-4d24-8c94-390faa43c342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:47.414653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 974/1960] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:47.427216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b759b29-9356-451a-a0d4-3ded597aee74 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:47.428989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7b759b29-9356-451a-a0d4-3ded597aee74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:47.490453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:47.495108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b759b29-9356-451a-a0d4-3ded597aee74 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:47.499765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de18555-c4ac-4cce-8460-f384d1506f98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:47.501389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18555-c4ac-4cce-8460-f384d1506f98 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:47.502192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18555-c4ac-4cce-8460-f384d1506f98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:47.502523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de18555-c4ac-4cce-8460-f384d1506f98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:47.564018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de18555-c4ac-4cce-8460-f384d1506f98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:47.565696 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de18555-c4ac-4cce-8460-f384d1506f98 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:47.575664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 987/1961] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:27:50.234500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8a9d6-679e-4d24-8c94-390faa43c342 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:50.250103 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d8a9d6-679e-4d24-8c94-390faa43c342 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:50.254828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216d80f-fa26-4fd3-97e2-20cae240bf27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:50.258470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216d80f-fa26-4fd3-97e2-20cae240bf27 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:50.258791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216d80f-fa26-4fd3-97e2-20cae240bf27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:50.259072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216d80f-fa26-4fd3-97e2-20cae240bf27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:50.264675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:50.289298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de18555-c4ac-4cce-8460-f384d1506f98 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:50.290820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8de18555-c4ac-4cce-8460-f384d1506f98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:50.352217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216d80f-fa26-4fd3-97e2-20cae240bf27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:50.353769 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216d80f-fa26-4fd3-97e2-20cae240bf27 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:50.366349 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 975/1962] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:27:50.386229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de18555-c4ac-4cce-8460-f384d1506f98 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:50.391462 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:50.392525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:50.392811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:50.393097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] PATH_INFO: `/v3/users/ec1bcac8a3ff47b981c6923e74d658d2` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:50.394420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec1bcac8a3ff47b981c6923e74d658d2)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:50.396501 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:50.470898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec1bcac8a3ff47b981c6923e74d658d2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:50.488415 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec1bcac8a3ff47b981c6923e74d658d2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:50.489170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec1bcac8a3ff47b981c6923e74d658d2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:50.498647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec1bcac8a3ff47b981c6923e74d658d2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:50.508711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec1bcac8a3ff47b981c6923e74d658d2'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:50.521375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 988/1963] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:27:50 2022] DELETE /identity/v3/users/ec1bcac8a3ff47b981c6923e74d658d2 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:50.528020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b61fcb-1b75-4ae3-ab94-645076bb1680 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:50.528255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b61fcb-1b75-4ae3-ab94-645076bb1680 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:50.528778 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b61fcb-1b75-4ae3-ab94-645076bb1680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:50.562968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e0b61fcb-1b75-4ae3-ab94-645076bb1680 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:50.565520 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e0b61fcb-1b75-4ae3-ab94-645076bb1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:50.691821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 976/1964] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:50.697230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:50.698733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:50.754769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:50.849099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81589ce9-5bc5-4d68-b570-53e3d2cc4a88 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:50.854827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6d1ff-befd-4296-9250-99d45585bfd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:50.856510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6d1ff-befd-4296-9250-99d45585bfd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:50.856916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6d1ff-befd-4296-9250-99d45585bfd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:50.857250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6d1ff-befd-4296-9250-99d45585bfd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:50.874328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6d1ff-befd-4296-9250-99d45585bfd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:50.876005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6d1ff-befd-4296-9250-99d45585bfd1 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:50.891390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 989/1965] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:27:51.318884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b61fcb-1b75-4ae3-ab94-645076bb1680 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:51.321364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e0b61fcb-1b75-4ae3-ab94-645076bb1680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:51.343454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b61fcb-1b75-4ae3-ab94-645076bb1680 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:51.349925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:51.351214 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:51.351701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:51.352132 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] PATH_INFO: `/v3/projects/6aff8fb6a7904019a3a05ea4aa2cc420` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:51.353768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aff8fb6a7904019a3a05ea4aa2cc420)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:51.356098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:51.357437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6d1ff-befd-4296-9250-99d45585bfd1 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:51.362836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9ee6d1ff-befd-4296-9250-99d45585bfd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:51.433337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aff8fb6a7904019a3a05ea4aa2cc420', 'request_id': 'req-2e019eb5-dce0-4e06-9f7c-a3ab33794755'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:51.440525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6d1ff-befd-4296-9250-99d45585bfd1 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:51.442689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aff8fb6a7904019a3a05ea4aa2cc420', 'request_id': 'req-2e019eb5-dce0-4e06-9f7c-a3ab33794755'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:51.452125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aff8fb6a7904019a3a05ea4aa2cc420', 'request_id': 'req-2e019eb5-dce0-4e06-9f7c-a3ab33794755'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:51.479683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] The token cache is being invalidate because project 6aff8fb6a7904019a3a05ea4aa2cc420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:27:51.479982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:51.480656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2e019eb5-dce0-4e06-9f7c-a3ab33794755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:51.482264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 977/1966] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:27:51 2022] DELETE /identity/v3/projects/6aff8fb6a7904019a3a05ea4aa2cc420 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:51.495449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f22e9a-48ce-4bf8-b667-6c0475aa0203 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:51.496230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f22e9a-48ce-4bf8-b667-6c0475aa0203 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:51.496629 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f22e9a-48ce-4bf8-b667-6c0475aa0203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:51.514511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1b7772-b70f-406d-a08e-69b781b21616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:51.516607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b7772-b70f-406d-a08e-69b781b21616 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:51.516904 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b7772-b70f-406d-a08e-69b781b21616 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:51.517193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b7772-b70f-406d-a08e-69b781b21616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:51.519692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bf1b7772-b70f-406d-a08e-69b781b21616 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:51.530206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c1f22e9a-48ce-4bf8-b667-6c0475aa0203 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:51.532530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c1f22e9a-48ce-4bf8-b667-6c0475aa0203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:51.564711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:51.596429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b7772-b70f-406d-a08e-69b781b21616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:51.598404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1b7772-b70f-406d-a08e-69b781b21616 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:51.611588 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 990/1967] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:51 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:27:51.616764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 978/1968] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:51.620169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391b387-c585-4c1d-881d-c867cd1fbc96 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:51.621251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391b387-c585-4c1d-881d-c867cd1fbc96 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:51.621251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391b387-c585-4c1d-881d-c867cd1fbc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:51.646919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a391b387-c585-4c1d-881d-c867cd1fbc96 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:51.648350 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a391b387-c585-4c1d-881d-c867cd1fbc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:51.658747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 991/1969] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:51.666436 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f22e9a-48ce-4bf8-b667-6c0475aa0203 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:51.684503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f22e9a-48ce-4bf8-b667-6c0475aa0203 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:51.691443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096cd10-5b3e-401a-8da4-0330d58af3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:51.692598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096cd10-5b3e-401a-8da4-0330d58af3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:51.693109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096cd10-5b3e-401a-8da4-0330d58af3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:51.693593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096cd10-5b3e-401a-8da4-0330d58af3f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:51.695067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096cd10-5b3e-401a-8da4-0330d58af3f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:27:51.695442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096cd10-5b3e-401a-8da4-0330d58af3f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:51.697382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096cd10-5b3e-401a-8da4-0330d58af3f4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:51.725520 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 979/1970] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:27:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:27:51.727109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391b387-c585-4c1d-881d-c867cd1fbc96 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:51.751076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391b387-c585-4c1d-881d-c867cd1fbc96 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:51.756557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2178a-59ad-494d-8362-685ea96c11db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:51.757500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2178a-59ad-494d-8362-685ea96c11db admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:51.757894 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2178a-59ad-494d-8362-685ea96c11db admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:51.758311 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2178a-59ad-494d-8362-685ea96c11db admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:51.759149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2178a-59ad-494d-8362-685ea96c11db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:51.761780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba2178a-59ad-494d-8362-685ea96c11db admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:51.802771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 992/1971] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:51 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:51.806936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096cd10-5b3e-401a-8da4-0330d58af3f4 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:51.817954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096cd10-5b3e-401a-8da4-0330d58af3f4 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:51.882953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd7cce-dc11-4ce3-9227-133fc2f622e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:51.883684 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd7cce-dc11-4ce3-9227-133fc2f622e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:51.883990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd7cce-dc11-4ce3-9227-133fc2f622e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:51.884248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd7cce-dc11-4ce3-9227-133fc2f622e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:51.884959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd7cce-dc11-4ce3-9227-133fc2f622e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:51.886949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd7cce-dc11-4ce3-9227-133fc2f622e1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:51.908817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 980/1972] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:51 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:51.913430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2178a-59ad-494d-8362-685ea96c11db admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:51.919300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:51.926511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2178a-59ad-494d-8362-685ea96c11db admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:51.932588 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfe9fd-f612-4c2e-8466-7cbc4bd34f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:51.933264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfe9fd-f612-4c2e-8466-7cbc4bd34f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:51.933548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfe9fd-f612-4c2e-8466-7cbc4bd34f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:51.933827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dfe9fd-f612-4c2e-8466-7cbc4bd34f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:51.934558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfe9fd-f612-4c2e-8466-7cbc4bd34f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:51.936209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dfe9fd-f612-4c2e-8466-7cbc4bd34f04 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:51.948111 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 993/1973] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:51.952262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd7cce-dc11-4ce3-9227-133fc2f622e1 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:51.963564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd7cce-dc11-4ce3-9227-133fc2f622e1 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:51.968861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746aced1-1e20-405c-aaa1-162909a00a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:51.969664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746aced1-1e20-405c-aaa1-162909a00a43 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:51.969830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746aced1-1e20-405c-aaa1-162909a00a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:51.970098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746aced1-1e20-405c-aaa1-162909a00a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:51.970883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746aced1-1e20-405c-aaa1-162909a00a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:51.972480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746aced1-1e20-405c-aaa1-162909a00a43 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:51.983008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 981/1974] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:51.988000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfe9fd-f612-4c2e-8466-7cbc4bd34f04 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:52.001604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dfe9fd-f612-4c2e-8466-7cbc4bd34f04 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:52.011874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cc9bb-fbae-4b86-b6ad-923155a68bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:52.012893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cc9bb-fbae-4b86-b6ad-923155a68bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:52.013299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cc9bb-fbae-4b86-b6ad-923155a68bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:52.013586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cc9bb-fbae-4b86-b6ad-923155a68bdf admin admin] PATH_INFO: `/v3/projects/604d62b88076405f91025f903a7d30ff/users/a2ec8bc242ca40dd891004b979c01f2e/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:52.035520 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cc9bb-fbae-4b86-b6ad-923155a68bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=604d62b88076405f91025f903a7d30ff, user_id=a2ec8bc242ca40dd891004b979c01f2e, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:52.039094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cc9bb-fbae-4b86-b6ad-923155a68bdf admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:52.061485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 994/1975] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:51 2022] PUT /identity/v3/projects/604d62b88076405f91025f903a7d30ff/users/a2ec8bc242ca40dd891004b979c01f2e/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:52.074783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d3509-4a51-4cd4-8c31-b1846ca54534 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:52.075588 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d3509-4a51-4cd4-8c31-b1846ca54534 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:52.076020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d3509-4a51-4cd4-8c31-b1846ca54534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:52.114136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-033d3509-4a51-4cd4-8c31-b1846ca54534 None None] MFA Rules not processed for user `a2ec8bc242ca40dd891004b979c01f2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:52.114806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-033d3509-4a51-4cd4-8c31-b1846ca54534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:52.198484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 982/1976] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:52.214571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cc9bb-fbae-4b86-b6ad-923155a68bdf admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:52.217545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-490cc9bb-fbae-4b86-b6ad-923155a68bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:52.307044 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cc9bb-fbae-4b86-b6ad-923155a68bdf admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:52.311249 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57198837-d824-4797-863c-5169559d7f8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:52.311960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57198837-d824-4797-863c-5169559d7f8c service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:52.312290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57198837-d824-4797-863c-5169559d7f8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:52.312575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57198837-d824-4797-863c-5169559d7f8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:52.325284 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57198837-d824-4797-863c-5169559d7f8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:52.326803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57198837-d824-4797-863c-5169559d7f8c service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:52.337587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 995/1977] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:27:56.396283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17211c4-9f45-4d61-b839-5eb8bd0a035d None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:56.399587 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17211c4-9f45-4d61-b839-5eb8bd0a035d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:56.400110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17211c4-9f45-4d61-b839-5eb8bd0a035d None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:56.405384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 983/1978] 10.222.0.64 () {54 vars in 921 bytes} [Wed Jul 20 08:27:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:27:56.418133 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a6581-c850-4bcc-8da2-0a7f3776b46f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:56.418681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a6581-c850-4bcc-8da2-0a7f3776b46f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:56.418681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a6581-c850-4bcc-8da2-0a7f3776b46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:56.441365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5e2a6581-c850-4bcc-8da2-0a7f3776b46f None None] MFA Rules not processed for user `928054a577d14522bd0f00417f4f8c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:56.443852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5e2a6581-c850-4bcc-8da2-0a7f3776b46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:56.536664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17211c4-9f45-4d61-b839-5eb8bd0a035d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:56.538031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e17211c4-9f45-4d61-b839-5eb8bd0a035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:56.553169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 996/1979] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Jul 20 08:27:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:56.567199 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a6581-c850-4bcc-8da2-0a7f3776b46f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:56.568744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5e2a6581-c850-4bcc-8da2-0a7f3776b46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:56.582225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a6581-c850-4bcc-8da2-0a7f3776b46f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:56.582902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:56.588648 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f255d2-9219-4358-94fb-6ae588030a9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:56.589341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f255d2-9219-4358-94fb-6ae588030a9c service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:56.589693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f255d2-9219-4358-94fb-6ae588030a9c service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:56.590103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f255d2-9219-4358-94fb-6ae588030a9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:56.600915 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f255d2-9219-4358-94fb-6ae588030a9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:56.602706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f255d2-9219-4358-94fb-6ae588030a9c service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:56.613247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 997/1980] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:27:56.631804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17211c4-9f45-4d61-b839-5eb8bd0a035d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:56.635948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec27ca-6b5f-4e01-a4ca-e198e8fb19e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:56.636822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec27ca-6b5f-4e01-a4ca-e198e8fb19e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:56.637105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec27ca-6b5f-4e01-a4ca-e198e8fb19e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:56.637408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec27ca-6b5f-4e01-a4ca-e198e8fb19e4 admin admin] PATH_INFO: `/v3/users/5bf1a059ddb34c2d8181712f53d592c2` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:56.638644 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec27ca-6b5f-4e01-a4ca-e198e8fb19e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf1a059ddb34c2d8181712f53d592c2)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:56.641136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec27ca-6b5f-4e01-a4ca-e198e8fb19e4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:56.705654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b9ec27ca-6b5f-4e01-a4ca-e198e8fb19e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf1a059ddb34c2d8181712f53d592c2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:56.707607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b9ec27ca-6b5f-4e01-a4ca-e198e8fb19e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf1a059ddb34c2d8181712f53d592c2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:56.717030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b9ec27ca-6b5f-4e01-a4ca-e198e8fb19e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf1a059ddb34c2d8181712f53d592c2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:56.733516 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b9ec27ca-6b5f-4e01-a4ca-e198e8fb19e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf1a059ddb34c2d8181712f53d592c2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:56.734602 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b9ec27ca-6b5f-4e01-a4ca-e198e8fb19e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf1a059ddb34c2d8181712f53d592c2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:56.740681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 984/1981] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:27:56 2022] DELETE /identity/v3/users/5bf1a059ddb34c2d8181712f53d592c2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:56.928779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f255d2-9219-4358-94fb-6ae588030a9c service placement] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:56.941562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f255d2-9219-4358-94fb-6ae588030a9c service placement] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:56.998732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:57.010268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:57.011190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:57.011481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:57.012248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] PATH_INFO: `/v3/projects/1b481fa73344413d85fc2eb5c0376c9d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:57.013059 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b481fa73344413d85fc2eb5c0376c9d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:57.015246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:57.066315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b481fa73344413d85fc2eb5c0376c9d', 'request_id': 'req-7c0e92e3-7ed3-4049-9829-f32d69522e20'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:57.073814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b481fa73344413d85fc2eb5c0376c9d', 'request_id': 'req-7c0e92e3-7ed3-4049-9829-f32d69522e20'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:57.081822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b481fa73344413d85fc2eb5c0376c9d', 'request_id': 'req-7c0e92e3-7ed3-4049-9829-f32d69522e20'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:57.092921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] The token cache is being invalidate because project 1b481fa73344413d85fc2eb5c0376c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:27:57.093285 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:57.094004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7c0e92e3-7ed3-4049-9829-f32d69522e20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:27:57.095609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 998/1982] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:27:56 2022] DELETE /identity/v3/projects/1b481fa73344413d85fc2eb5c0376c9d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:57.100962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031be8f6-2718-47e2-a027-e86f81acb2d9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:57.101319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031be8f6-2718-47e2-a027-e86f81acb2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:57.101666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031be8f6-2718-47e2-a027-e86f81acb2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:57.120424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-031be8f6-2718-47e2-a027-e86f81acb2d9 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:57.121673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-031be8f6-2718-47e2-a027-e86f81acb2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:57.187516 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 985/1983] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:57.195698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97554979-06ca-4a61-83a8-dfe33bc23718 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:57.195698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97554979-06ca-4a61-83a8-dfe33bc23718 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:57.196352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97554979-06ca-4a61-83a8-dfe33bc23718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:57.218513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-97554979-06ca-4a61-83a8-dfe33bc23718 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:57.219785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-97554979-06ca-4a61-83a8-dfe33bc23718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:57.227699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 999/1984] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:27:57.231993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031be8f6-2718-47e2-a027-e86f81acb2d9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:57.243276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031be8f6-2718-47e2-a027-e86f81acb2d9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:57.248449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c918bc-4534-4927-b004-40ef4738e829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:57.249114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c918bc-4534-4927-b004-40ef4738e829 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:57.249431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c918bc-4534-4927-b004-40ef4738e829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:57.250255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c918bc-4534-4927-b004-40ef4738e829 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:57.251029 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c918bc-4534-4927-b004-40ef4738e829 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:27:57.251343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c918bc-4534-4927-b004-40ef4738e829 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:57.253223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c918bc-4534-4927-b004-40ef4738e829 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:57.266321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 986/1985] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:27:57.270915 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97554979-06ca-4a61-83a8-dfe33bc23718 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:57.280841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97554979-06ca-4a61-83a8-dfe33bc23718 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:57.286344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b6055-cbf8-4fb5-9f26-71396d1a328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:57.287059 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b6055-cbf8-4fb5-9f26-71396d1a328c admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:57.287341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b6055-cbf8-4fb5-9f26-71396d1a328c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:57.287615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b6055-cbf8-4fb5-9f26-71396d1a328c admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:57.288318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b6055-cbf8-4fb5-9f26-71396d1a328c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:57.290255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b6055-cbf8-4fb5-9f26-71396d1a328c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:57.321256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1000/1986] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:27:57 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:57.325613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c918bc-4534-4927-b004-40ef4738e829 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:57.335938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c918bc-4534-4927-b004-40ef4738e829 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:57.368877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:27:57.404261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950341b-5873-4788-a914-ef61141ea380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:57.404941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950341b-5873-4788-a914-ef61141ea380 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:57.405212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950341b-5873-4788-a914-ef61141ea380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:57.405488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950341b-5873-4788-a914-ef61141ea380 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:57.406389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950341b-5873-4788-a914-ef61141ea380 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:57.408353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950341b-5873-4788-a914-ef61141ea380 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:57.430620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 987/1987] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:27:57 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:27:57.435089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b6055-cbf8-4fb5-9f26-71396d1a328c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:57.449979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b6055-cbf8-4fb5-9f26-71396d1a328c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:57.455374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19283a3a-93c5-4833-b8a4-6cada6807677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:57.456050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283a3a-93c5-4833-b8a4-6cada6807677 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:57.456337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283a3a-93c5-4833-b8a4-6cada6807677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:57.456615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19283a3a-93c5-4833-b8a4-6cada6807677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:57.457359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19283a3a-93c5-4833-b8a4-6cada6807677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:57.459121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19283a3a-93c5-4833-b8a4-6cada6807677 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:57.472324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1001/1988] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:27:57.476689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950341b-5873-4788-a914-ef61141ea380 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:57.490692 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950341b-5873-4788-a914-ef61141ea380 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:57.495824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce377a5-8a46-42aa-b5c3-d672a6eddfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:57.496509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce377a5-8a46-42aa-b5c3-d672a6eddfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:57.496798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce377a5-8a46-42aa-b5c3-d672a6eddfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:57.497081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce377a5-8a46-42aa-b5c3-d672a6eddfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:57.497817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce377a5-8a46-42aa-b5c3-d672a6eddfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:57.500182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce377a5-8a46-42aa-b5c3-d672a6eddfeb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:57.511562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 988/1989] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:27:57.516276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19283a3a-93c5-4833-b8a4-6cada6807677 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:57.529474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19283a3a-93c5-4833-b8a4-6cada6807677 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:57.534956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2728a-8fd9-4d02-981e-176040f14e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:57.536012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2728a-8fd9-4d02-981e-176040f14e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:57.536375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2728a-8fd9-4d02-981e-176040f14e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:57.536846 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2728a-8fd9-4d02-981e-176040f14e8b admin admin] PATH_INFO: `/v3/projects/330b802242fb4f0aafbea5d6967cbc23/users/dc2f6b545dda4089998a8cb99932ce16/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:57.564163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2728a-8fd9-4d02-981e-176040f14e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=330b802242fb4f0aafbea5d6967cbc23, user_id=dc2f6b545dda4089998a8cb99932ce16, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:57.570181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2728a-8fd9-4d02-981e-176040f14e8b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:57.595005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1002/1990] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:27:57 2022] PUT /identity/v3/projects/330b802242fb4f0aafbea5d6967cbc23/users/dc2f6b545dda4089998a8cb99932ce16/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:27:57.612423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2ce1a-c70a-4034-a37a-7d24bedb1b57 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:57.612726 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2ce1a-c70a-4034-a37a-7d24bedb1b57 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:57.613039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2ce1a-c70a-4034-a37a-7d24bedb1b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:57.646731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cbc2ce1a-c70a-4034-a37a-7d24bedb1b57 None None] MFA Rules not processed for user `dc2f6b545dda4089998a8cb99932ce16`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:27:57.648684 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cbc2ce1a-c70a-4034-a37a-7d24bedb1b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:57.756282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 989/1991] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:27:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:27:57.771737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2728a-8fd9-4d02-981e-176040f14e8b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:27:57.773559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-32f2728a-8fd9-4d02-981e-176040f14e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:27:57.844898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2728a-8fd9-4d02-981e-176040f14e8b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:27:57.848689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95df5f6-df69-4b1b-a20a-671fece17eb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:27:57.849290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df5f6-df69-4b1b-a20a-671fece17eb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:27:57.849575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df5f6-df69-4b1b-a20a-671fece17eb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:27:57.849853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95df5f6-df69-4b1b-a20a-671fece17eb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:27:57.860118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95df5f6-df69-4b1b-a20a-671fece17eb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:27:57.861604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95df5f6-df69-4b1b-a20a-671fece17eb0 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:27:57.872313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1003/1992] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:27:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:28:02.061998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2ce1a-c70a-4034-a37a-7d24bedb1b57 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:02.065781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cbc2ce1a-c70a-4034-a37a-7d24bedb1b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:02.159303 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2ce1a-c70a-4034-a37a-7d24bedb1b57 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:02.166426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:02.167282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:02.167610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:02.167935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] PATH_INFO: `/v3/users/956adb80f4ae4d2ea759afab00fccd68` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:02.169574 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=956adb80f4ae4d2ea759afab00fccd68)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:02.172529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:02.238037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:02.244827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '956adb80f4ae4d2ea759afab00fccd68'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:02.258904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '956adb80f4ae4d2ea759afab00fccd68'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:02.271934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '956adb80f4ae4d2ea759afab00fccd68'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:02.288482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '956adb80f4ae4d2ea759afab00fccd68'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:02.289242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '956adb80f4ae4d2ea759afab00fccd68'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:02.295324 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 990/1993] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:28:02 2022] DELETE /identity/v3/users/956adb80f4ae4d2ea759afab00fccd68 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:02.302439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5806a-d65d-4cd9-8b88-b10bad7916ea None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:02.303009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5806a-d65d-4cd9-8b88-b10bad7916ea None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:02.303009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e5806a-d65d-4cd9-8b88-b10bad7916ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:02.322604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-94e5806a-d65d-4cd9-8b88-b10bad7916ea None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:02.323892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-94e5806a-d65d-4cd9-8b88-b10bad7916ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:02.345702 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:02.406712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1004/1994] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:28:02.417770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:02.419651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:02.512503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a4e63-707f-4b12-8421-b131766d2ae5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:02.518943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa47f2-ae62-491f-a717-9a0c8b3a7636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:02.519618 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa47f2-ae62-491f-a717-9a0c8b3a7636 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:02.519901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa47f2-ae62-491f-a717-9a0c8b3a7636 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:02.520178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa47f2-ae62-491f-a717-9a0c8b3a7636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:02.548183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa47f2-ae62-491f-a717-9a0c8b3a7636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:02.550875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa47f2-ae62-491f-a717-9a0c8b3a7636 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:02.564399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 991/1995] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:28:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:28:02.879282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5806a-d65d-4cd9-8b88-b10bad7916ea None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:02.880995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-94e5806a-d65d-4cd9-8b88-b10bad7916ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:02.894910 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e5806a-d65d-4cd9-8b88-b10bad7916ea None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:02.900006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:02.900936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:02.901223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:02.901507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] PATH_INFO: `/v3/projects/f50eb3e07f4746ca84d4610e2d339dc8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:02.902792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50eb3e07f4746ca84d4610e2d339dc8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:02.904954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:02.967482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50eb3e07f4746ca84d4610e2d339dc8', 'request_id': 'req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:02.976076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50eb3e07f4746ca84d4610e2d339dc8', 'request_id': 'req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:02.983006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50eb3e07f4746ca84d4610e2d339dc8', 'request_id': 'req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:02.993958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] The token cache is being invalidate because project f50eb3e07f4746ca84d4610e2d339dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:28:02.994247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:02.994944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:02.996505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1005/1996] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:28:02 2022] DELETE /identity/v3/projects/f50eb3e07f4746ca84d4610e2d339dc8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:03.001293 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa47f2-ae62-491f-a717-9a0c8b3a7636 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:03.012770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa47f2-ae62-491f-a717-9a0c8b3a7636 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:03.076613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:03.078255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:03.078605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:03.078969 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] PATH_INFO: `/v3/users/059c35b81a6f4388b215e53c6e24a2b6` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:03.080418 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=059c35b81a6f4388b215e53c6e24a2b6)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:03.082378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:03.137035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '059c35b81a6f4388b215e53c6e24a2b6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.145809 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '059c35b81a6f4388b215e53c6e24a2b6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.154957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '059c35b81a6f4388b215e53c6e24a2b6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.157860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:03.172907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '059c35b81a6f4388b215e53c6e24a2b6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.173798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '059c35b81a6f4388b215e53c6e24a2b6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.181014 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 992/1997] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:28:03 2022] DELETE /identity/v3/users/059c35b81a6f4388b215e53c6e24a2b6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:28:03.538122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:03.540304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:03.568252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d4e3d-3ef7-4ecc-91d1-15cb9c22a8fb admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:03.655157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:03.656617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:03.657027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:03.657563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] PATH_INFO: `/v3/projects/ce2942b959b641d1bc0dde5437f9a094` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:03.659184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2942b959b641d1bc0dde5437f9a094)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:03.661282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:03.723491 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:03.725668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2942b959b641d1bc0dde5437f9a094', 'request_id': 'req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.737278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2942b959b641d1bc0dde5437f9a094', 'request_id': 'req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.745472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2942b959b641d1bc0dde5437f9a094', 'request_id': 'req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.759491 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] The token cache is being invalidate because project ce2942b959b641d1bc0dde5437f9a094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:28:03.759820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.760767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.762694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1006/1998] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:28:03 2022] DELETE /identity/v3/projects/ce2942b959b641d1bc0dde5437f9a094 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:03.768000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:03.770300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:03.791068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e31207-ccbd-4b36-ae1d-d0cb8c1b5f95 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:03.863894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:03.864817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:03.865101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:03.865552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] PATH_INFO: `/v3/users/dadd52dcf4a4489eb0c0fea5b80ba772` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:03.867024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dadd52dcf4a4489eb0c0fea5b80ba772)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:03.869109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:03.940804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dadd52dcf4a4489eb0c0fea5b80ba772'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.950794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dadd52dcf4a4489eb0c0fea5b80ba772'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.961972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dadd52dcf4a4489eb0c0fea5b80ba772'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.981789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dadd52dcf4a4489eb0c0fea5b80ba772'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.982792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dadd52dcf4a4489eb0c0fea5b80ba772'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:03.991431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 993/1999] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:28:03 2022] DELETE /identity/v3/users/dadd52dcf4a4489eb0c0fea5b80ba772 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:04.362600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:04.364839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:04.392465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:04.394405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:04.412252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a5cc8-d1bd-48aa-b000-21005f8e4b80 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:04.496018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eeb859-817c-4bb2-bba3-92e92850d404 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:04.504528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e68673-6b54-4fdb-bff1-3bc0c87e527f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:04.506336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e68673-6b54-4fdb-bff1-3bc0c87e527f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:04.506834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e68673-6b54-4fdb-bff1-3bc0c87e527f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:04.507295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e68673-6b54-4fdb-bff1-3bc0c87e527f admin admin] PATH_INFO: `/v3/users/a2ec8bc242ca40dd891004b979c01f2e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:04.509662 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e68673-6b54-4fdb-bff1-3bc0c87e527f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2ec8bc242ca40dd891004b979c01f2e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:04.513273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e68673-6b54-4fdb-bff1-3bc0c87e527f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:04.514523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:04.515602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:04.516088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:04.516387 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:04.517641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] PATH_INFO: `/v3/projects/236af605fe7a4609a6d31f6654b6b346` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:04.520174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=236af605fe7a4609a6d31f6654b6b346)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:04.524133 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:04.589454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c2e68673-6b54-4fdb-bff1-3bc0c87e527f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2ec8bc242ca40dd891004b979c01f2e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:04.590643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '236af605fe7a4609a6d31f6654b6b346', 'request_id': 'req-04b0e512-adb8-4e26-bf9c-28271c39c035'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:04.603506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c2e68673-6b54-4fdb-bff1-3bc0c87e527f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2ec8bc242ca40dd891004b979c01f2e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:04.604594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '236af605fe7a4609a6d31f6654b6b346', 'request_id': 'req-04b0e512-adb8-4e26-bf9c-28271c39c035'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:04.617174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c2e68673-6b54-4fdb-bff1-3bc0c87e527f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2ec8bc242ca40dd891004b979c01f2e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:04.617404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '236af605fe7a4609a6d31f6654b6b346', 'request_id': 'req-04b0e512-adb8-4e26-bf9c-28271c39c035'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:04.636782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c2e68673-6b54-4fdb-bff1-3bc0c87e527f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2ec8bc242ca40dd891004b979c01f2e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:04.637644 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c2e68673-6b54-4fdb-bff1-3bc0c87e527f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2ec8bc242ca40dd891004b979c01f2e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:04.637831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] The token cache is being invalidate because project 236af605fe7a4609a6d31f6654b6b346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:28:04.638223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:04.639292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-04b0e512-adb8-4e26-bf9c-28271c39c035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:04.642123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1007/2000] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:28:04 2022] DELETE /identity/v3/projects/236af605fe7a4609a6d31f6654b6b346 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:04.648034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a4524-9ac5-4f81-b084-5c0656d94ac4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:04.648323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a4524-9ac5-4f81-b084-5c0656d94ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:04.648616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a4524-9ac5-4f81-b084-5c0656d94ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:04.654398 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 994/2001] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:28:04 2022] DELETE /identity/v3/users/a2ec8bc242ca40dd891004b979c01f2e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:04.657652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e6a7b-98e6-43f0-998b-cea4a19d555a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:04.657652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e6a7b-98e6-43f0-998b-cea4a19d555a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:04.657926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e6a7b-98e6-43f0-998b-cea4a19d555a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:04.678371 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-992a4524-9ac5-4f81-b084-5c0656d94ac4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:04.680116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-992a4524-9ac5-4f81-b084-5c0656d94ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:04.683852 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9f7e6a7b-98e6-43f0-998b-cea4a19d555a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:04.685107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9f7e6a7b-98e6-43f0-998b-cea4a19d555a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:04.776114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 995/2002] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:28:04.787535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e6a7b-98e6-43f0-998b-cea4a19d555a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:04.789205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9f7e6a7b-98e6-43f0-998b-cea4a19d555a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:04.807569 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1008/2003] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:28:04.818463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b824a-457e-4c40-9a5a-e55742d94add None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:04.819424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b824a-457e-4c40-9a5a-e55742d94add None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:04.820462 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b824a-457e-4c40-9a5a-e55742d94add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:04.868258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:04.870405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e6a7b-98e6-43f0-998b-cea4a19d555a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:04.885853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ca339-546f-4eda-8a6d-b04323d42404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:04.885853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ca339-546f-4eda-8a6d-b04323d42404 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:04.885853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ca339-546f-4eda-8a6d-b04323d42404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:04.885853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ca339-546f-4eda-8a6d-b04323d42404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:04.891372 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e95b824a-457e-4c40-9a5a-e55742d94add None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:04.893179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ca339-546f-4eda-8a6d-b04323d42404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:04.893845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e95b824a-457e-4c40-9a5a-e55742d94add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:04.894837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ca339-546f-4eda-8a6d-b04323d42404 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:04.908822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 996/2004] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:28:04.912613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1009/2005] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:28:04.918031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ca339-546f-4eda-8a6d-b04323d42404 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:04.938865 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ca339-546f-4eda-8a6d-b04323d42404 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:04.948105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da475a-1f6a-4e85-80ad-1dc5146569a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:04.949711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da475a-1f6a-4e85-80ad-1dc5146569a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:04.949887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da475a-1f6a-4e85-80ad-1dc5146569a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:04.950502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da475a-1f6a-4e85-80ad-1dc5146569a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:04.953362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da475a-1f6a-4e85-80ad-1dc5146569a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:28:04.954496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da475a-1f6a-4e85-80ad-1dc5146569a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:04.958205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da475a-1f6a-4e85-80ad-1dc5146569a8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:04.978662 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 997/2006] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:28:04.984010 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b824a-457e-4c40-9a5a-e55742d94add None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:05.003749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b824a-457e-4c40-9a5a-e55742d94add None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:05.013990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ced95-b5b4-49b1-a3cf-94bf9cb61ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:05.015929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ced95-b5b4-49b1-a3cf-94bf9cb61ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.016259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ced95-b5b4-49b1-a3cf-94bf9cb61ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.017097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ced95-b5b4-49b1-a3cf-94bf9cb61ccd admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.018757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ced95-b5b4-49b1-a3cf-94bf9cb61ccd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:05.023357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ced95-b5b4-49b1-a3cf-94bf9cb61ccd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:05.074707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1010/2007] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:28:04 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:28:05.079977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da475a-1f6a-4e85-80ad-1dc5146569a8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:05.091436 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da475a-1f6a-4e85-80ad-1dc5146569a8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:05.100474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:05.184911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731313c9-0e34-4aa3-a772-50b5cf84750a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:05.185733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731313c9-0e34-4aa3-a772-50b5cf84750a admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.186112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731313c9-0e34-4aa3-a772-50b5cf84750a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.186514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731313c9-0e34-4aa3-a772-50b5cf84750a admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.187965 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731313c9-0e34-4aa3-a772-50b5cf84750a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:05.190200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731313c9-0e34-4aa3-a772-50b5cf84750a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:05.220164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 998/2008] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:28:05 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:28:05.221672 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ced95-b5b4-49b1-a3cf-94bf9cb61ccd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:05.233237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ced95-b5b4-49b1-a3cf-94bf9cb61ccd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:05.238685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6473e-dda7-4e01-b232-7b5bfa730da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:05.239399 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6473e-dda7-4e01-b232-7b5bfa730da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.239680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6473e-dda7-4e01-b232-7b5bfa730da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.239961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6473e-dda7-4e01-b232-7b5bfa730da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.240726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6473e-dda7-4e01-b232-7b5bfa730da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:05.242389 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6473e-dda7-4e01-b232-7b5bfa730da2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:05.259266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1011/2009] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:28:05.263753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731313c9-0e34-4aa3-a772-50b5cf84750a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:05.275513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731313c9-0e34-4aa3-a772-50b5cf84750a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:05.281700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2161276-1071-47b4-8b65-94e8eb56f564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:05.282345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161276-1071-47b4-8b65-94e8eb56f564 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.282809 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161276-1071-47b4-8b65-94e8eb56f564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.282971 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2161276-1071-47b4-8b65-94e8eb56f564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.283699 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2161276-1071-47b4-8b65-94e8eb56f564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:05.285432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2161276-1071-47b4-8b65-94e8eb56f564 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:05.299073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 999/2010] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:28:05.302755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6473e-dda7-4e01-b232-7b5bfa730da2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:05.318422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6473e-dda7-4e01-b232-7b5bfa730da2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:05.326761 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c182-478f-49fc-b03e-c10957b6e623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:05.328381 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c182-478f-49fc-b03e-c10957b6e623 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.328381 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c182-478f-49fc-b03e-c10957b6e623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.329673 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c182-478f-49fc-b03e-c10957b6e623 admin admin] PATH_INFO: `/v3/projects/a9e2a2795c4b4532903b3c301000d47f/users/2434d7353e51423e8d3244838eccf020/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.362774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63c182-478f-49fc-b03e-c10957b6e623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e2a2795c4b4532903b3c301000d47f, user_id=2434d7353e51423e8d3244838eccf020, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:05.369038 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63c182-478f-49fc-b03e-c10957b6e623 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:05.399683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1012/2011] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:28:05 2022] PUT /identity/v3/projects/a9e2a2795c4b4532903b3c301000d47f/users/2434d7353e51423e8d3244838eccf020/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:05.411083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9474e4-1639-4897-9128-98275c895b5a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.412363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9474e4-1639-4897-9128-98275c895b5a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.413100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9474e4-1639-4897-9128-98275c895b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.452640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-fd9474e4-1639-4897-9128-98275c895b5a None None] MFA Rules not processed for user `2434d7353e51423e8d3244838eccf020`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:05.455566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fd9474e4-1639-4897-9128-98275c895b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:05.480165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c182-478f-49fc-b03e-c10957b6e623 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:05.483552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eb63c182-478f-49fc-b03e-c10957b6e623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:05.579072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1000/2012] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:28:05.595581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9474e4-1639-4897-9128-98275c895b5a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:05.603337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fd9474e4-1639-4897-9128-98275c895b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:05.607433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c182-478f-49fc-b03e-c10957b6e623 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:05.614177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:05.615109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.615647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.616142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] PATH_INFO: `/v3/projects/604d62b88076405f91025f903a7d30ff` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.617759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=604d62b88076405f91025f903a7d30ff)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:05.620088 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:05.692466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '604d62b88076405f91025f903a7d30ff', 'request_id': 'req-ac7a80c4-3e9a-4037-9507-214976d52896'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:05.720516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '604d62b88076405f91025f903a7d30ff', 'request_id': 'req-ac7a80c4-3e9a-4037-9507-214976d52896'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:05.732942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '604d62b88076405f91025f903a7d30ff', 'request_id': 'req-ac7a80c4-3e9a-4037-9507-214976d52896'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:05.748820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] The token cache is being invalidate because project 604d62b88076405f91025f903a7d30ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:28:05.749320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:05.750434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ac7a80c4-3e9a-4037-9507-214976d52896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:05.752708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1013/2013] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:28:05 2022] DELETE /identity/v3/projects/604d62b88076405f91025f903a7d30ff => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:28:05.757526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d77d24-7a1b-4c0f-ae60-6fd80813825c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.757526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d77d24-7a1b-4c0f-ae60-6fd80813825c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.757526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d77d24-7a1b-4c0f-ae60-6fd80813825c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.759733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9474e4-1639-4897-9128-98275c895b5a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:05.796598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-10d77d24-7a1b-4c0f-ae60-6fd80813825c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:05.797948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-10d77d24-7a1b-4c0f-ae60-6fd80813825c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:05.817420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:05.834302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:05.854144 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50ee23-1c31-419c-86c0-212b9d303786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:05.855336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50ee23-1c31-419c-86c0-212b9d303786 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.855460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50ee23-1c31-419c-86c0-212b9d303786 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.856048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50ee23-1c31-419c-86c0-212b9d303786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.858349 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bb50ee23-1c31-419c-86c0-212b9d303786 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:05.878638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1014/2014] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:28:05.887379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8642f1-cb2c-408b-9b96-6d6f29342da6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.887871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8642f1-cb2c-408b-9b96-6d6f29342da6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.888244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8642f1-cb2c-408b-9b96-6d6f29342da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.926320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5c8642f1-cb2c-408b-9b96-6d6f29342da6 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:05.926320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5c8642f1-cb2c-408b-9b96-6d6f29342da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:05.934733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1015/2015] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:28:05.937765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8642f1-cb2c-408b-9b96-6d6f29342da6 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:05.950973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8642f1-cb2c-408b-9b96-6d6f29342da6 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:05.961621 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b095779-cfa5-4321-b9d3-3b938527cac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:05.962311 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b095779-cfa5-4321-b9d3-3b938527cac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.962627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b095779-cfa5-4321-b9d3-3b938527cac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.962881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b095779-cfa5-4321-b9d3-3b938527cac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.963743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b095779-cfa5-4321-b9d3-3b938527cac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:28:05.964051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b095779-cfa5-4321-b9d3-3b938527cac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:05.965783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b095779-cfa5-4321-b9d3-3b938527cac4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:05.968137 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50ee23-1c31-419c-86c0-212b9d303786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:05.976674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50ee23-1c31-419c-86c0-212b9d303786 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:05.979738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1016/2016] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:28:05.981102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4350-f4bf-403f-b4ea-fdbaf6ac305f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:05.981393 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4350-f4bf-403f-b4ea-fdbaf6ac305f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:05.981705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d4350-f4bf-403f-b4ea-fdbaf6ac305f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:05.996011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1001/2017] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:28:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:28:05.997818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50ee23-1c31-419c-86c0-212b9d303786 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:06.008352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e25d4350-f4bf-403f-b4ea-fdbaf6ac305f None None] MFA Rules not processed for user `d88c5f05aab0458d8dc0a285bce33f28`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:06.009688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e25d4350-f4bf-403f-b4ea-fdbaf6ac305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:06.016879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50ee23-1c31-419c-86c0-212b9d303786 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:06.030893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076af60c-7279-4538-9670-17db8f1a15be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:06.031701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076af60c-7279-4538-9670-17db8f1a15be admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:06.032107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076af60c-7279-4538-9670-17db8f1a15be admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:06.032280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076af60c-7279-4538-9670-17db8f1a15be admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:06.033236 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076af60c-7279-4538-9670-17db8f1a15be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:06.035336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076af60c-7279-4538-9670-17db8f1a15be admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:06.089588 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1002/2018] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:28:05 2022] POST /identity/v3/projects => generated 367 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:28:06.092315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076af60c-7279-4538-9670-17db8f1a15be admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:06.143587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1017/2019] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:28:06.192257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076af60c-7279-4538-9670-17db8f1a15be admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:06.208897 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c4720-dcc0-4291-bfff-362c95e70de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:06.210152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c4720-dcc0-4291-bfff-362c95e70de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:06.210664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c4720-dcc0-4291-bfff-362c95e70de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:06.211377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c4720-dcc0-4291-bfff-362c95e70de8 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:06.214246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c4720-dcc0-4291-bfff-362c95e70de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:06.222001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c4720-dcc0-4291-bfff-362c95e70de8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:06.268194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1003/2020] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:28:06 2022] POST /identity/v3/users => generated 284 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:28:06.272540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d4350-f4bf-403f-b4ea-fdbaf6ac305f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:06.295426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d4350-f4bf-403f-b4ea-fdbaf6ac305f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:06.309113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d0dd8-b3c4-4d79-9c9d-d027b5c9a4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:06.310263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d0dd8-b3c4-4d79-9c9d-d027b5c9a4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:06.311017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d0dd8-b3c4-4d79-9c9d-d027b5c9a4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:06.311516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d0dd8-b3c4-4d79-9c9d-d027b5c9a4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:06.313603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d0dd8-b3c4-4d79-9c9d-d027b5c9a4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:06.316128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4d0dd8-b3c4-4d79-9c9d-d027b5c9a4fa admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:06.341705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1018/2021] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:28:06.346842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c4720-dcc0-4291-bfff-362c95e70de8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:06.372233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c4720-dcc0-4291-bfff-362c95e70de8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:06.380845 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beb9791-3b3f-4c31-8b5e-1d7a8c68e7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:06.381957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb9791-3b3f-4c31-8b5e-1d7a8c68e7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:06.382508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb9791-3b3f-4c31-8b5e-1d7a8c68e7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:06.383309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beb9791-3b3f-4c31-8b5e-1d7a8c68e7a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:06.384592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb9791-3b3f-4c31-8b5e-1d7a8c68e7a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:06.386475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beb9791-3b3f-4c31-8b5e-1d7a8c68e7a6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:06.420424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1004/2022] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:28:06.427433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d0dd8-b3c4-4d79-9c9d-d027b5c9a4fa admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:06.438854 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d0dd8-b3c4-4d79-9c9d-d027b5c9a4fa admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:06.445217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547be852-056e-4fe2-8509-e0f2767e0a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:06.445217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547be852-056e-4fe2-8509-e0f2767e0a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:06.445217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547be852-056e-4fe2-8509-e0f2767e0a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:06.445593 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547be852-056e-4fe2-8509-e0f2767e0a88 admin admin] PATH_INFO: `/v3/projects/f10167a86c1e4f0e97f215966e218de3/users/ad10b9c3a2d643aab6ed68587d1dcc1c/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:06.466169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547be852-056e-4fe2-8509-e0f2767e0a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f10167a86c1e4f0e97f215966e218de3, user_id=ad10b9c3a2d643aab6ed68587d1dcc1c, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:06.470634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547be852-056e-4fe2-8509-e0f2767e0a88 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:06.492811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1019/2023] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:28:06 2022] PUT /identity/v3/projects/f10167a86c1e4f0e97f215966e218de3/users/ad10b9c3a2d643aab6ed68587d1dcc1c/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:06.504995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e30bc-1ee7-4ff5-9acf-6b68f7fd912a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:06.505395 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e30bc-1ee7-4ff5-9acf-6b68f7fd912a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:06.505594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e30bc-1ee7-4ff5-9acf-6b68f7fd912a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:06.540154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-563e30bc-1ee7-4ff5-9acf-6b68f7fd912a None None] MFA Rules not processed for user `ad10b9c3a2d643aab6ed68587d1dcc1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:06.541926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-563e30bc-1ee7-4ff5-9acf-6b68f7fd912a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:06.601534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:06.662819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1005/2024] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:06 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:28:06.677532 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547be852-056e-4fe2-8509-e0f2767e0a88 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:06.680340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-547be852-056e-4fe2-8509-e0f2767e0a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:06.805968 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547be852-056e-4fe2-8509-e0f2767e0a88 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:06.822270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f20a2-5bbc-4a72-9260-ea42ffb395df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3778997232d44cd3a3c4fb3d53e4db53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:06.822963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f20a2-5bbc-4a72-9260-ea42ffb395df service glance] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:06.823224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f20a2-5bbc-4a72-9260-ea42ffb395df service glance] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:06.823691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f20a2-5bbc-4a72-9260-ea42ffb395df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:06.856174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f20a2-5bbc-4a72-9260-ea42ffb395df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:06.858421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f20a2-5bbc-4a72-9260-ea42ffb395df service glance] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:06.871587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1020/2025] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:28:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:28:11.431194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e30bc-1ee7-4ff5-9acf-6b68f7fd912a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:11.435246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-563e30bc-1ee7-4ff5-9acf-6b68f7fd912a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:11.559905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e30bc-1ee7-4ff5-9acf-6b68f7fd912a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:11.566493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:11.567455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:11.567790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:11.568030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] PATH_INFO: `/v3/users/dc2f6b545dda4089998a8cb99932ce16` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:11.569369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2f6b545dda4089998a8cb99932ce16)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:11.571542 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:11.659400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2f6b545dda4089998a8cb99932ce16'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:11.676052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2f6b545dda4089998a8cb99932ce16'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:11.686669 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2f6b545dda4089998a8cb99932ce16'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:11.708672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2f6b545dda4089998a8cb99932ce16'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:11.709698 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2f6b545dda4089998a8cb99932ce16'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:11.722306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1006/2026] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:28:11 2022] DELETE /identity/v3/users/dc2f6b545dda4089998a8cb99932ce16 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:11.727596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941ea13-deb9-4d25-8d64-e3357cec0ce7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:11.727596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941ea13-deb9-4d25-8d64-e3357cec0ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:11.727596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2941ea13-deb9-4d25-8d64-e3357cec0ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:11.750396 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2941ea13-deb9-4d25-8d64-e3357cec0ce7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:11.751783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2941ea13-deb9-4d25-8d64-e3357cec0ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:11.783424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:11.828911 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1021/2027] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:28:11.846775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:11.848341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:11.939764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5315d11-69e6-45d3-a15a-47d8b260874f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:11.943529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870cd62-c4ae-46ed-bf88-29972abbc3ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:11.944227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870cd62-c4ae-46ed-bf88-29972abbc3ca service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:11.944593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870cd62-c4ae-46ed-bf88-29972abbc3ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:11.944800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870cd62-c4ae-46ed-bf88-29972abbc3ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:11.952348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:11.957200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870cd62-c4ae-46ed-bf88-29972abbc3ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:11.959723 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870cd62-c4ae-46ed-bf88-29972abbc3ca service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:11.971359 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1007/2028] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:28:12.433071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941ea13-deb9-4d25-8d64-e3357cec0ce7 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:12.434628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2941ea13-deb9-4d25-8d64-e3357cec0ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:12.464919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2941ea13-deb9-4d25-8d64-e3357cec0ce7 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:12.478375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:12.479969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:12.480743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:12.481200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] PATH_INFO: `/v3/projects/330b802242fb4f0aafbea5d6967cbc23` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:12.483504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] RBAC: Authorizing `identity:delete_project(project_id=330b802242fb4f0aafbea5d6967cbc23)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:12.487167 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:12.551266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '330b802242fb4f0aafbea5d6967cbc23', 'request_id': 'req-6740afdc-edaf-4e20-b0a6-9e0765c02210'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:12.566117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '330b802242fb4f0aafbea5d6967cbc23', 'request_id': 'req-6740afdc-edaf-4e20-b0a6-9e0765c02210'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:12.576410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '330b802242fb4f0aafbea5d6967cbc23', 'request_id': 'req-6740afdc-edaf-4e20-b0a6-9e0765c02210'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:12.589879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] The token cache is being invalidate because project 330b802242fb4f0aafbea5d6967cbc23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:28:12.590302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:12.591057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6740afdc-edaf-4e20-b0a6-9e0765c02210 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:12.593685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1022/2029] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:28:12 2022] DELETE /identity/v3/projects/330b802242fb4f0aafbea5d6967cbc23 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:12.599609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221157cd-a2cd-4fb0-947e-297cbfdafed1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:12.600158 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221157cd-a2cd-4fb0-947e-297cbfdafed1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:12.601074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221157cd-a2cd-4fb0-947e-297cbfdafed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:12.627823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-221157cd-a2cd-4fb0-947e-297cbfdafed1 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:12.629167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-221157cd-a2cd-4fb0-947e-297cbfdafed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:12.703666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1008/2030] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:28:12.713608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248146ca-b9f3-43f4-ab44-1a9995babad2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:12.713906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248146ca-b9f3-43f4-ab44-1a9995babad2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:12.714198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248146ca-b9f3-43f4-ab44-1a9995babad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:12.759009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-248146ca-b9f3-43f4-ab44-1a9995babad2 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:12.761171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-248146ca-b9f3-43f4-ab44-1a9995babad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:12.776809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1023/2031] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:28:12.781633 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221157cd-a2cd-4fb0-947e-297cbfdafed1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:12.792885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221157cd-a2cd-4fb0-947e-297cbfdafed1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:12.798071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4f19-7f96-4213-9836-279571cefaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:12.798782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4f19-7f96-4213-9836-279571cefaa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:12.799072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4f19-7f96-4213-9836-279571cefaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:12.799352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eb4f19-7f96-4213-9836-279571cefaa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:12.800093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb4f19-7f96-4213-9836-279571cefaa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:28:12.800407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb4f19-7f96-4213-9836-279571cefaa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:12.802037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eb4f19-7f96-4213-9836-279571cefaa3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:12.815819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1009/2032] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:28:12.820544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248146ca-b9f3-43f4-ab44-1a9995babad2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:12.832819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248146ca-b9f3-43f4-ab44-1a9995babad2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:12.838320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc9bfa-9100-4d11-a956-42869cd540cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:12.839090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc9bfa-9100-4d11-a956-42869cd540cd admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:12.839378 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc9bfa-9100-4d11-a956-42869cd540cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:12.839664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc9bfa-9100-4d11-a956-42869cd540cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:12.840403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc9bfa-9100-4d11-a956-42869cd540cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:12.842649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc9bfa-9100-4d11-a956-42869cd540cd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:12.879557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1024/2033] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:28:12 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:28:12.881192 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4f19-7f96-4213-9836-279571cefaa3 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:12.893297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eb4f19-7f96-4213-9836-279571cefaa3 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:13.011277 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17d82-3f16-4e08-9e06-08c9482499da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:13.011987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17d82-3f16-4e08-9e06-08c9482499da admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:13.012276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17d82-3f16-4e08-9e06-08c9482499da admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:13.012557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc17d82-3f16-4e08-9e06-08c9482499da admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:13.013267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc17d82-3f16-4e08-9e06-08c9482499da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:13.016102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc17d82-3f16-4e08-9e06-08c9482499da admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:13.047282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1010/2034] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:28:12 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:28:13.052723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc9bfa-9100-4d11-a956-42869cd540cd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:13.064123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc9bfa-9100-4d11-a956-42869cd540cd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:13.070629 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03de1d-b487-4665-a622-c528f609e2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:13.072205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03de1d-b487-4665-a622-c528f609e2fd admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:13.073084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03de1d-b487-4665-a622-c528f609e2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:13.073897 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03de1d-b487-4665-a622-c528f609e2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:13.075376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03de1d-b487-4665-a622-c528f609e2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:13.078944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03de1d-b487-4665-a622-c528f609e2fd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:13.100051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1025/2035] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:28:13.104638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17d82-3f16-4e08-9e06-08c9482499da admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:13.118610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc17d82-3f16-4e08-9e06-08c9482499da admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:13.123624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5aeec-29ab-4def-b78d-a7a261329244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:13.124570 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5aeec-29ab-4def-b78d-a7a261329244 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:13.124727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5aeec-29ab-4def-b78d-a7a261329244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:13.125081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5aeec-29ab-4def-b78d-a7a261329244 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:13.125775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5aeec-29ab-4def-b78d-a7a261329244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:13.127809 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5aeec-29ab-4def-b78d-a7a261329244 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:13.139120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1011/2036] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:28:13.144172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03de1d-b487-4665-a622-c528f609e2fd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:13.154481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03de1d-b487-4665-a622-c528f609e2fd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:13.159970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39048b0e-9bb8-44db-b7d6-2c0e3fafa213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:13.160861 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39048b0e-9bb8-44db-b7d6-2c0e3fafa213 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:13.161139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39048b0e-9bb8-44db-b7d6-2c0e3fafa213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:13.161426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39048b0e-9bb8-44db-b7d6-2c0e3fafa213 admin admin] PATH_INFO: `/v3/projects/ef27aa2b591747229fbca4a581606b1b/users/90708052de6c4c9083efe0d494217048/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:13.182442 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39048b0e-9bb8-44db-b7d6-2c0e3fafa213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef27aa2b591747229fbca4a581606b1b, user_id=90708052de6c4c9083efe0d494217048, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:13.189145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39048b0e-9bb8-44db-b7d6-2c0e3fafa213 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:13.207212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1026/2037] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:28:13 2022] PUT /identity/v3/projects/ef27aa2b591747229fbca4a581606b1b/users/90708052de6c4c9083efe0d494217048/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:13.225924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42226557-1632-4b16-9ccb-305da29df378 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:13.226524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42226557-1632-4b16-9ccb-305da29df378 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:13.227024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42226557-1632-4b16-9ccb-305da29df378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:13.258516 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-42226557-1632-4b16-9ccb-305da29df378 None None] MFA Rules not processed for user `90708052de6c4c9083efe0d494217048`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:13.260202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-42226557-1632-4b16-9ccb-305da29df378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:13.334069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:13.355941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1012/2038] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:28:13.361631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39048b0e-9bb8-44db-b7d6-2c0e3fafa213 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:13.418387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:13.445421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39048b0e-9bb8-44db-b7d6-2c0e3fafa213 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:13.451875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4744cd1-e9ab-4165-8f79-94fea1634cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:13.452716 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4744cd1-e9ab-4165-8f79-94fea1634cdf admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:13.453106 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4744cd1-e9ab-4165-8f79-94fea1634cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:13.453713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4744cd1-e9ab-4165-8f79-94fea1634cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:13.454263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4744cd1-e9ab-4165-8f79-94fea1634cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:13.456320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4744cd1-e9ab-4165-8f79-94fea1634cdf admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:13.498550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1027/2039] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:28:13 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:28:13.501793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42226557-1632-4b16-9ccb-305da29df378 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:13.518923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42226557-1632-4b16-9ccb-305da29df378 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:13.616891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397754d-4879-4aba-b049-baf89f81b405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:13.618009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397754d-4879-4aba-b049-baf89f81b405 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:13.618306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397754d-4879-4aba-b049-baf89f81b405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:13.618950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397754d-4879-4aba-b049-baf89f81b405 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:13.620379 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397754d-4879-4aba-b049-baf89f81b405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:13.624893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397754d-4879-4aba-b049-baf89f81b405 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:13.664777 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1013/2040] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:28:13 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:28:13.667025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4744cd1-e9ab-4165-8f79-94fea1634cdf admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:13.681902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4744cd1-e9ab-4165-8f79-94fea1634cdf admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:13.688661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64eaf9-8b4d-4563-8484-809f45c299a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:13.689853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64eaf9-8b4d-4563-8484-809f45c299a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:13.690380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64eaf9-8b4d-4563-8484-809f45c299a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:13.690865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d64eaf9-8b4d-4563-8484-809f45c299a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:13.691926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64eaf9-8b4d-4563-8484-809f45c299a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:13.694076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d64eaf9-8b4d-4563-8484-809f45c299a0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:13.715364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1028/2041] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:28:13.717725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397754d-4879-4aba-b049-baf89f81b405 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:13.728202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397754d-4879-4aba-b049-baf89f81b405 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:13.737740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71f7b1-7f70-47f8-b40d-0c2d780b662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:13.738717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71f7b1-7f70-47f8-b40d-0c2d780b662f admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:13.739638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71f7b1-7f70-47f8-b40d-0c2d780b662f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:13.739638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71f7b1-7f70-47f8-b40d-0c2d780b662f admin admin] PATH_INFO: `/v3/projects/d9ccd187f98f4560a3c65f6119e11b3b/users/8a08dd96e2814d6daa47bc472db7da43/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:13.762180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71f7b1-7f70-47f8-b40d-0c2d780b662f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ccd187f98f4560a3c65f6119e11b3b, user_id=8a08dd96e2814d6daa47bc472db7da43, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:13.768188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71f7b1-7f70-47f8-b40d-0c2d780b662f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:13.801979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1014/2042] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:28:13 2022] PUT /identity/v3/projects/d9ccd187f98f4560a3c65f6119e11b3b/users/8a08dd96e2814d6daa47bc472db7da43/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:13.804355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64eaf9-8b4d-4563-8484-809f45c299a0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:13.822565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d64eaf9-8b4d-4563-8484-809f45c299a0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:13.940828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10d5c5-b927-41e4-ad8d-4c6c10fb717a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:13.942090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10d5c5-b927-41e4-ad8d-4c6c10fb717a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:13.942737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10d5c5-b927-41e4-ad8d-4c6c10fb717a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:13.943333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10d5c5-b927-41e4-ad8d-4c6c10fb717a admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:13.944837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10d5c5-b927-41e4-ad8d-4c6c10fb717a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:13.948089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10d5c5-b927-41e4-ad8d-4c6c10fb717a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:13.974836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1029/2043] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:28:13.977886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71f7b1-7f70-47f8-b40d-0c2d780b662f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:13.995261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71f7b1-7f70-47f8-b40d-0c2d780b662f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:14.025364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af46bd8-a763-4c43-afaa-cee7dc4a65cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:14.026471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af46bd8-a763-4c43-afaa-cee7dc4a65cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:14.026934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af46bd8-a763-4c43-afaa-cee7dc4a65cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:14.027418 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af46bd8-a763-4c43-afaa-cee7dc4a65cd admin admin] PATH_INFO: `/v3/projects/d9ccd187f98f4560a3c65f6119e11b3b/users/8a08dd96e2814d6daa47bc472db7da43/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:14.029739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af46bd8-a763-4c43-afaa-cee7dc4a65cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ccd187f98f4560a3c65f6119e11b3b, user_id=8a08dd96e2814d6daa47bc472db7da43, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:14.035860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af46bd8-a763-4c43-afaa-cee7dc4a65cd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:14.052418 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1015/2044] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:28:13 2022] PUT /identity/v3/projects/d9ccd187f98f4560a3c65f6119e11b3b/users/8a08dd96e2814d6daa47bc472db7da43/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:14.064825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff76bf-4143-4cdd-a738-2714109c1ed5 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:14.065165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff76bf-4143-4cdd-a738-2714109c1ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:14.065419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff76bf-4143-4cdd-a738-2714109c1ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:14.110280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-19ff76bf-4143-4cdd-a738-2714109c1ed5 None None] MFA Rules not processed for user `8a08dd96e2814d6daa47bc472db7da43`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:14.111655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-19ff76bf-4143-4cdd-a738-2714109c1ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:14.209206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1030/2045] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:28:14.221283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af46bd8-a763-4c43-afaa-cee7dc4a65cd admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:14.223052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8af46bd8-a763-4c43-afaa-cee7dc4a65cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:14.310815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af46bd8-a763-4c43-afaa-cee7dc4a65cd admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:14.315416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bd05a-b576-429a-a511-fbb90127b43e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:14.316444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bd05a-b576-429a-a511-fbb90127b43e service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:14.316917 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bd05a-b576-429a-a511-fbb90127b43e service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:14.317293 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bd05a-b576-429a-a511-fbb90127b43e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:14.338625 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bd05a-b576-429a-a511-fbb90127b43e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:14.340165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bd05a-b576-429a-a511-fbb90127b43e service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:14.356102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1016/2046] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:28:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:28:14.469515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff76bf-4143-4cdd-a738-2714109c1ed5 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:14.480012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff76bf-4143-4cdd-a738-2714109c1ed5 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:14.484738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d7933-1ada-4f5b-a743-464e124b6f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:14.485372 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d7933-1ada-4f5b-a743-464e124b6f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:14.485646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d7933-1ada-4f5b-a743-464e124b6f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:14.485917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d7933-1ada-4f5b-a743-464e124b6f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:14.554407 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:14.583927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d7933-1ada-4f5b-a743-464e124b6f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:14.585699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d7933-1ada-4f5b-a743-464e124b6f6e service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:14.598661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1031/2047] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:28:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:28:24.348493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:24.592611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=9 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:30.462480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bd05a-b576-429a-a511-fbb90127b43e service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:30.464254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f78bd05a-b576-429a-a511-fbb90127b43e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:30.534681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bd05a-b576-429a-a511-fbb90127b43e service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:30.539608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:30.540528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:30.540816 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:30.541093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] PATH_INFO: `/v3/users/2434d7353e51423e8d3244838eccf020` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:30.542323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2434d7353e51423e8d3244838eccf020)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:30.544368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:30.596333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2434d7353e51423e8d3244838eccf020'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:30.603354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2434d7353e51423e8d3244838eccf020'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:30.612364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2434d7353e51423e8d3244838eccf020'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:30.628703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2434d7353e51423e8d3244838eccf020'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:30.629430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2434d7353e51423e8d3244838eccf020'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:30.635651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1017/2048] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:28:30 2022] DELETE /identity/v3/users/2434d7353e51423e8d3244838eccf020 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:28:30.640113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f33272-23b9-4209-ac9e-708ef9559494 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:30.640603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f33272-23b9-4209-ac9e-708ef9559494 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:30.640717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f33272-23b9-4209-ac9e-708ef9559494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:30.661954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-40f33272-23b9-4209-ac9e-708ef9559494 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:30.663247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-40f33272-23b9-4209-ac9e-708ef9559494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:30.733484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1032/2049] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:28:30.745821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:30.748074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:30.817287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41a595-2550-4e33-bcfd-2430ab02a1f6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:30.821754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2019f0-c128-467a-9c2f-16b34bde9557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:30.822528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2019f0-c128-467a-9c2f-16b34bde9557 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:30.822878 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2019f0-c128-467a-9c2f-16b34bde9557 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:30.823201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2019f0-c128-467a-9c2f-16b34bde9557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:30.835070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2019f0-c128-467a-9c2f-16b34bde9557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:30.836510 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2019f0-c128-467a-9c2f-16b34bde9557 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:30.846547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1018/2050] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:28:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:28:31.136944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f33272-23b9-4209-ac9e-708ef9559494 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:31.138592 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-40f33272-23b9-4209-ac9e-708ef9559494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:31.159855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f33272-23b9-4209-ac9e-708ef9559494 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:31.166623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:31.168286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:31.168701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:31.169047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] PATH_INFO: `/v3/projects/a9e2a2795c4b4532903b3c301000d47f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:31.171193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e2a2795c4b4532903b3c301000d47f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:31.173551 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:31.226628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e2a2795c4b4532903b3c301000d47f', 'request_id': 'req-41596c16-d2be-4161-8844-c5093c94eedf'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:31.236430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e2a2795c4b4532903b3c301000d47f', 'request_id': 'req-41596c16-d2be-4161-8844-c5093c94eedf'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:31.244827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e2a2795c4b4532903b3c301000d47f', 'request_id': 'req-41596c16-d2be-4161-8844-c5093c94eedf'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:31.258198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] The token cache is being invalidate because project a9e2a2795c4b4532903b3c301000d47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:28:31.258198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:31.258530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:28:31.260263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1033/2051] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:28:31 2022] DELETE /identity/v3/projects/a9e2a2795c4b4532903b3c301000d47f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:28:33.280533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce3104-0541-445b-8b15-6fddb6a1c69d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:33.281068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce3104-0541-445b-8b15-6fddb6a1c69d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:33.281162 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce3104-0541-445b-8b15-6fddb6a1c69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:33.307470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7dce3104-0541-445b-8b15-6fddb6a1c69d None None] MFA Rules not processed for user `bdc9e59d03334b72a8052e0f1112523f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:28:33.308721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7dce3104-0541-445b-8b15-6fddb6a1c69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:33.328032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:33.371290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1019/2052] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:28:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:28:41.253906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:43.364190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=15 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:28:55.839487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:28:55.840926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:55.913056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41596c16-d2be-4161-8844-c5093c94eedf admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:28:55.918253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2637c2-fd5a-4ddf-af0e-14b9faa59b97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:28:55.918941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2637c2-fd5a-4ddf-af0e-14b9faa59b97 service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:28:55.919229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2637c2-fd5a-4ddf-af0e-14b9faa59b97 service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:28:55.919510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2637c2-fd5a-4ddf-af0e-14b9faa59b97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:28:55.921301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eb2637c2-fd5a-4ddf-af0e-14b9faa59b97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:28:55.934009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2637c2-fd5a-4ddf-af0e-14b9faa59b97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:28:55.935685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2637c2-fd5a-4ddf-af0e-14b9faa59b97 service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:28:55.946270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1034/2053] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:28:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:05.942141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=32 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:14.983357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cb143-8776-4e66-bf90-b988cd7a2607 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:14.983851 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cb143-8776-4e66-bf90-b988cd7a2607 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:14.983940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790cb143-8776-4e66-bf90-b988cd7a2607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:15.008091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-790cb143-8776-4e66-bf90-b988cd7a2607 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:15.008091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-790cb143-8776-4e66-bf90-b988cd7a2607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:15.060865 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1020/2054] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:29:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:15.102536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2637c2-fd5a-4ddf-af0e-14b9faa59b97 service placement] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:15.104177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eb2637c2-fd5a-4ddf-af0e-14b9faa59b97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:15.176494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2637c2-fd5a-4ddf-af0e-14b9faa59b97 service placement] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:15.181470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d186f78-7bec-43e0-bfcb-66e800b270dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:15.182227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d186f78-7bec-43e0-bfcb-66e800b270dc service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:15.182519 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d186f78-7bec-43e0-bfcb-66e800b270dc service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:15.182828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d186f78-7bec-43e0-bfcb-66e800b270dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:15.192392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d186f78-7bec-43e0-bfcb-66e800b270dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:15.194058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d186f78-7bec-43e0-bfcb-66e800b270dc service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:15.202450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1035/2055] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:29:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 08:29:25.054155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=25 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:25.199175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=32 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:33.591060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cb143-8776-4e66-bf90-b988cd7a2607 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:33.592986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-790cb143-8776-4e66-bf90-b988cd7a2607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:33.729284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790cb143-8776-4e66-bf90-b988cd7a2607 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:33.742668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46b205-2516-4b76-bbe4-b0ff458e57e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:33.744011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46b205-2516-4b76-bbe4-b0ff458e57e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:33.744503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46b205-2516-4b76-bbe4-b0ff458e57e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:33.744985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46b205-2516-4b76-bbe4-b0ff458e57e5 admin admin] PATH_INFO: `/v3/users/ad10b9c3a2d643aab6ed68587d1dcc1c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:33.747075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46b205-2516-4b76-bbe4-b0ff458e57e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad10b9c3a2d643aab6ed68587d1dcc1c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:33.751065 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a46b205-2516-4b76-bbe4-b0ff458e57e5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:33.812013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d186f78-7bec-43e0-bfcb-66e800b270dc service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:33.814140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6d186f78-7bec-43e0-bfcb-66e800b270dc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:33.835482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d186f78-7bec-43e0-bfcb-66e800b270dc service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:33.841446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad308219-ba09-48f6-9288-c01957a1a513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:33.843316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad308219-ba09-48f6-9288-c01957a1a513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:33.844111 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad308219-ba09-48f6-9288-c01957a1a513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:33.844729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad308219-ba09-48f6-9288-c01957a1a513 admin admin] PATH_INFO: `/v3/users/90708052de6c4c9083efe0d494217048` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:33.846570 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad308219-ba09-48f6-9288-c01957a1a513 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90708052de6c4c9083efe0d494217048)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:33.849479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad308219-ba09-48f6-9288-c01957a1a513 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:33.856508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3a46b205-2516-4b76-bbe4-b0ff458e57e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad10b9c3a2d643aab6ed68587d1dcc1c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:33.871813 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3a46b205-2516-4b76-bbe4-b0ff458e57e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad10b9c3a2d643aab6ed68587d1dcc1c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:33.891991 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3a46b205-2516-4b76-bbe4-b0ff458e57e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad10b9c3a2d643aab6ed68587d1dcc1c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:33.930656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3a46b205-2516-4b76-bbe4-b0ff458e57e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad10b9c3a2d643aab6ed68587d1dcc1c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:33.931477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3a46b205-2516-4b76-bbe4-b0ff458e57e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad10b9c3a2d643aab6ed68587d1dcc1c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:33.939823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ad308219-ba09-48f6-9288-c01957a1a513 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90708052de6c4c9083efe0d494217048'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:33.943947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1021/2056] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:29:33 2022] DELETE /identity/v3/users/ad10b9c3a2d643aab6ed68587d1dcc1c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:33.948683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2535d2-80ab-46fc-9342-107461caf8b2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:33.949010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2535d2-80ab-46fc-9342-107461caf8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:33.949264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2535d2-80ab-46fc-9342-107461caf8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:33.964287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ad308219-ba09-48f6-9288-c01957a1a513 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90708052de6c4c9083efe0d494217048'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:33.965586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ad308219-ba09-48f6-9288-c01957a1a513 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90708052de6c4c9083efe0d494217048'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:33.973805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-be2535d2-80ab-46fc-9342-107461caf8b2 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:33.975274 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-be2535d2-80ab-46fc-9342-107461caf8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:33.977250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ad308219-ba09-48f6-9288-c01957a1a513 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90708052de6c4c9083efe0d494217048'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:33.990691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ad308219-ba09-48f6-9288-c01957a1a513 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90708052de6c4c9083efe0d494217048'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:33.999433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1036/2057] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:29:33 2022] DELETE /identity/v3/users/90708052de6c4c9083efe0d494217048 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:34.006344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6433f-41f6-4544-a4dc-7b5a0a5f2a86 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:34.006344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6433f-41f6-4544-a4dc-7b5a0a5f2a86 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:34.006683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6433f-41f6-4544-a4dc-7b5a0a5f2a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:34.056204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-23b6433f-41f6-4544-a4dc-7b5a0a5f2a86 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:34.058227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-23b6433f-41f6-4544-a4dc-7b5a0a5f2a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:34.071179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1022/2058] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:29:34.088513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2535d2-80ab-46fc-9342-107461caf8b2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:34.091389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-be2535d2-80ab-46fc-9342-107461caf8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:34.151352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:34.176142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1037/2059] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:34.177804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6433f-41f6-4544-a4dc-7b5a0a5f2a86 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:34.179308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-23b6433f-41f6-4544-a4dc-7b5a0a5f2a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:34.208280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2535d2-80ab-46fc-9342-107461caf8b2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:34.283240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6433f-41f6-4544-a4dc-7b5a0a5f2a86 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:34.287979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad68c00-f0fa-4b6c-bddc-b7c9ca97e0be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:34.288740 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad68c00-f0fa-4b6c-bddc-b7c9ca97e0be service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:34.289116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad68c00-f0fa-4b6c-bddc-b7c9ca97e0be service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:34.289639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad68c00-f0fa-4b6c-bddc-b7c9ca97e0be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:34.303430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad68c00-f0fa-4b6c-bddc-b7c9ca97e0be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:34.305465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad68c00-f0fa-4b6c-bddc-b7c9ca97e0be service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:34.310504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b25ad-9584-4e2c-aec0-080ccffc9831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:34.312402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b25ad-9584-4e2c-aec0-080ccffc9831 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:34.312818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b25ad-9584-4e2c-aec0-080ccffc9831 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:34.313239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b25ad-9584-4e2c-aec0-080ccffc9831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:34.326171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b25ad-9584-4e2c-aec0-080ccffc9831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:34.327805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b25ad-9584-4e2c-aec0-080ccffc9831 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:34.337947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1038/2060] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:34.353858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1023/2061] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:34.752526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad68c00-f0fa-4b6c-bddc-b7c9ca97e0be service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:34.755767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cad68c00-f0fa-4b6c-bddc-b7c9ca97e0be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:34.784771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad68c00-f0fa-4b6c-bddc-b7c9ca97e0be service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:34.801900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:34.803236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:34.812848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:34.813427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] PATH_INFO: `/v3/projects/ef27aa2b591747229fbca4a581606b1b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:34.814408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b25ad-9584-4e2c-aec0-080ccffc9831 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:34.815624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef27aa2b591747229fbca4a581606b1b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:34.816048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0b0b25ad-9584-4e2c-aec0-080ccffc9831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:34.818233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:34.848833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b25ad-9584-4e2c-aec0-080ccffc9831 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:34.852639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:34.861991 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:34.863023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:34.863541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:34.864117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] PATH_INFO: `/v3/projects/f10167a86c1e4f0e97f215966e218de3` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:34.867924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f10167a86c1e4f0e97f215966e218de3)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:34.871458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:34.934281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef27aa2b591747229fbca4a581606b1b', 'request_id': 'req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:34.947474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef27aa2b591747229fbca4a581606b1b', 'request_id': 'req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:34.959445 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef27aa2b591747229fbca4a581606b1b', 'request_id': 'req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:34.978782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] The token cache is being invalidate because project ef27aa2b591747229fbca4a581606b1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:29:34.979397 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:34.980094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:34.982619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1039/2062] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:29:34 2022] DELETE /identity/v3/projects/ef27aa2b591747229fbca4a581606b1b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:34.987376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:34.989386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:35.008238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f10167a86c1e4f0e97f215966e218de3', 'request_id': 'req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:35.026740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f10167a86c1e4f0e97f215966e218de3', 'request_id': 'req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:35.034762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f10167a86c1e4f0e97f215966e218de3', 'request_id': 'req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:35.060002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] The token cache is being invalidate because project f10167a86c1e4f0e97f215966e218de3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:29:35.060388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:35.061507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f8a6719c-8117-42ae-b20b-9eae1ec5ce7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:35.063905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1024/2063] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:29:34 2022] DELETE /identity/v3/projects/f10167a86c1e4f0e97f215966e218de3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:35.077556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5038be-72a3-4749-987c-e244f80faf40 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:35.078461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5038be-72a3-4749-987c-e244f80faf40 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:35.079748 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5038be-72a3-4749-987c-e244f80faf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:35.131448 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-df5038be-72a3-4749-987c-e244f80faf40 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:35.134527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-df5038be-72a3-4749-987c-e244f80faf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:35.134981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980e630-9f46-44fd-a42c-dd1b4e38b2e9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:35.247074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:35.248009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:35.248251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:35.248535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] PATH_INFO: `/v3/users/8a08dd96e2814d6daa47bc472db7da43` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:35.250174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a08dd96e2814d6daa47bc472db7da43)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:35.252461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:35.257805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1025/2064] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:35.266538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21c0a8-b20f-42f9-b62a-3568f75416fd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:35.267276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21c0a8-b20f-42f9-b62a-3568f75416fd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:35.267866 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21c0a8-b20f-42f9-b62a-3568f75416fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:35.276802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:35.295137 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6f21c0a8-b20f-42f9-b62a-3568f75416fd None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:35.296382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6f21c0a8-b20f-42f9-b62a-3568f75416fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:35.306829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1026/2065] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:35.312456 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21c0a8-b20f-42f9-b62a-3568f75416fd None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:35.326966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21c0a8-b20f-42f9-b62a-3568f75416fd None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:35.334011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2cbf9-5ca5-43cf-be49-8c268ca53324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:35.334739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2cbf9-5ca5-43cf-be49-8c268ca53324 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:35.335592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2cbf9-5ca5-43cf-be49-8c268ca53324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:35.335891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2cbf9-5ca5-43cf-be49-8c268ca53324 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:35.336650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2cbf9-5ca5-43cf-be49-8c268ca53324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:29:35.337044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2cbf9-5ca5-43cf-be49-8c268ca53324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:35.338802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2cbf9-5ca5-43cf-be49-8c268ca53324 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:35.343490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a08dd96e2814d6daa47bc472db7da43'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:35.348069 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:35.356612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1027/2066] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:29:35.361487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2cbf9-5ca5-43cf-be49-8c268ca53324 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:35.373803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a08dd96e2814d6daa47bc472db7da43'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:35.374771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a08dd96e2814d6daa47bc472db7da43'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:35.377370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2cbf9-5ca5-43cf-be49-8c268ca53324 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:35.383403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd5db5-9159-4e99-956b-bb2c0dc6ef7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:35.384154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd5db5-9159-4e99-956b-bb2c0dc6ef7f admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:35.384525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd5db5-9159-4e99-956b-bb2c0dc6ef7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:35.384921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd5db5-9159-4e99-956b-bb2c0dc6ef7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:35.385711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd5db5-9159-4e99-956b-bb2c0dc6ef7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:35.387918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd5db5-9159-4e99-956b-bb2c0dc6ef7f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:35.406496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a08dd96e2814d6daa47bc472db7da43'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:35.422981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a08dd96e2814d6daa47bc472db7da43'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:35.431331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1028/2067] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:29:35 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:29:35.432847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd5db5-9159-4e99-956b-bb2c0dc6ef7f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:35.443738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1040/2068] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:29:34 2022] DELETE /identity/v3/users/8a08dd96e2814d6daa47bc472db7da43 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:29:35.443950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-11cd5db5-9159-4e99-956b-bb2c0dc6ef7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:35.553384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd5db5-9159-4e99-956b-bb2c0dc6ef7f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:35.673470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c6836-8c7f-4f47-b55d-2174e012926b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:35.674461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c6836-8c7f-4f47-b55d-2174e012926b admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:35.674964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c6836-8c7f-4f47-b55d-2174e012926b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:35.675420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c6836-8c7f-4f47-b55d-2174e012926b admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:35.676331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c6836-8c7f-4f47-b55d-2174e012926b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:35.678573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c6836-8c7f-4f47-b55d-2174e012926b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:35.711442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1029/2069] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:29:35 2022] POST /identity/v3/users => generated 286 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:35.717216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:35.736408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e726a6-d87b-4990-b453-4d97aa7e2811 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:35.745431 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab11fd5-6c51-41c1-9add-6f57c429ee1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:35.746844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab11fd5-6c51-41c1-9add-6f57c429ee1a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:35.747960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab11fd5-6c51-41c1-9add-6f57c429ee1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:35.748665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab11fd5-6c51-41c1-9add-6f57c429ee1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:35.750244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab11fd5-6c51-41c1-9add-6f57c429ee1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:35.752698 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab11fd5-6c51-41c1-9add-6f57c429ee1a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:35.777984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1041/2070] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:35.783062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c6836-8c7f-4f47-b55d-2174e012926b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:35.803950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c6836-8c7f-4f47-b55d-2174e012926b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:35.808998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f5a81-9e99-4f0b-a5a5-28eedb9e86bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:35.809882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f5a81-9e99-4f0b-a5a5-28eedb9e86bd admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:35.810180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f5a81-9e99-4f0b-a5a5-28eedb9e86bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:35.810467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f5a81-9e99-4f0b-a5a5-28eedb9e86bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:35.811260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f5a81-9e99-4f0b-a5a5-28eedb9e86bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:35.812946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f5a81-9e99-4f0b-a5a5-28eedb9e86bd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:35.841904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1030/2071] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:35 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:29:35.843544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab11fd5-6c51-41c1-9add-6f57c429ee1a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:35.856631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab11fd5-6c51-41c1-9add-6f57c429ee1a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:35.864534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0188b-5539-446b-9355-58341bdc2199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:35.865648 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0188b-5539-446b-9355-58341bdc2199 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:35.865800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0188b-5539-446b-9355-58341bdc2199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:35.866512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0188b-5539-446b-9355-58341bdc2199 admin admin] PATH_INFO: `/v3/projects/df0009d53ec14eb48ad8dbe662d5e1d1/users/4018a77d5ea4425daf25f03957d3557d/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:35.899652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0188b-5539-446b-9355-58341bdc2199 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0009d53ec14eb48ad8dbe662d5e1d1, user_id=4018a77d5ea4425daf25f03957d3557d, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:35.904137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0188b-5539-446b-9355-58341bdc2199 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:35.909954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f5a81-9e99-4f0b-a5a5-28eedb9e86bd admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:35.912797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6d8f5a81-9e99-4f0b-a5a5-28eedb9e86bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:35.939533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f5a81-9e99-4f0b-a5a5-28eedb9e86bd admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:35.958631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1042/2072] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:35 2022] PUT /identity/v3/projects/df0009d53ec14eb48ad8dbe662d5e1d1/users/4018a77d5ea4425daf25f03957d3557d/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:35.969676 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0ab7a-7a5e-4f2a-825f-6aeeb001cc6c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:35.970377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0ab7a-7a5e-4f2a-825f-6aeeb001cc6c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:35.971061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0ab7a-7a5e-4f2a-825f-6aeeb001cc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.027381 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:36.038360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-73d0ab7a-7a5e-4f2a-825f-6aeeb001cc6c None None] MFA Rules not processed for user `4018a77d5ea4425daf25f03957d3557d`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:36.039874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-73d0ab7a-7a5e-4f2a-825f-6aeeb001cc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:36.044299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:36.045973 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.046374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.046827 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] PATH_INFO: `/v3/projects/d9ccd187f98f4560a3c65f6119e11b3b` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.048386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ccd187f98f4560a3c65f6119e11b3b)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:36.052096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:36.135819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ccd187f98f4560a3c65f6119e11b3b', 'request_id': 'req-5121b668-4cc2-47e6-96b7-90f4079025f7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:36.143349 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ccd187f98f4560a3c65f6119e11b3b', 'request_id': 'req-5121b668-4cc2-47e6-96b7-90f4079025f7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:36.151071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ccd187f98f4560a3c65f6119e11b3b', 'request_id': 'req-5121b668-4cc2-47e6-96b7-90f4079025f7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:36.167230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] The token cache is being invalidate because project d9ccd187f98f4560a3c65f6119e11b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:29:36.167587 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:36.168336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:36.172962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1043/2073] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:29:36.173150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1031/2074] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:29:35 2022] DELETE /identity/v3/projects/d9ccd187f98f4560a3c65f6119e11b3b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:29:36.179782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe083a0-d3b8-4894-89e1-0c1130836206 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.180217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe083a0-d3b8-4894-89e1-0c1130836206 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.180571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe083a0-d3b8-4894-89e1-0c1130836206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.182484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:36.184075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:36.204065 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ebe083a0-d3b8-4894-89e1-0c1130836206 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:36.205502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ebe083a0-d3b8-4894-89e1-0c1130836206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:36.280057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121b668-4cc2-47e6-96b7-90f4079025f7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:36.284763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ad620-19f9-4941-acd7-74e3f1b415ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:36.285427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ad620-19f9-4941-acd7-74e3f1b415ee service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.285713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ad620-19f9-4941-acd7-74e3f1b415ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.286003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ad620-19f9-4941-acd7-74e3f1b415ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.287817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bc9ad620-19f9-4941-acd7-74e3f1b415ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:36.292035 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1044/2075] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:29:36.302161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b60701-4c77-4847-9db3-1eeeec0d5933 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.302453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b60701-4c77-4847-9db3-1eeeec0d5933 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.302781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b60701-4c77-4847-9db3-1eeeec0d5933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.328962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-85b60701-4c77-4847-9db3-1eeeec0d5933 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:36.330823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-85b60701-4c77-4847-9db3-1eeeec0d5933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:36.344036 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1045/2076] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:36.349968 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b60701-4c77-4847-9db3-1eeeec0d5933 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:36.368488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b60701-4c77-4847-9db3-1eeeec0d5933 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:36.374247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2eef0-6919-4739-aa1f-b23f89338138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:36.376363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2eef0-6919-4739-aa1f-b23f89338138 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.376885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2eef0-6919-4739-aa1f-b23f89338138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.377392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2eef0-6919-4739-aa1f-b23f89338138 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.378188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2eef0-6919-4739-aa1f-b23f89338138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:29:36.378518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2eef0-6919-4739-aa1f-b23f89338138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:36.380944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2eef0-6919-4739-aa1f-b23f89338138 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:36.398969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:36.404367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1046/2077] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:29:36.410902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2eef0-6919-4739-aa1f-b23f89338138 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:36.425550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2eef0-6919-4739-aa1f-b23f89338138 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:36.432699 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ad620-19f9-4941-acd7-74e3f1b415ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:36.433474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5adf9-0c8c-44a5-81be-ea8f62a13b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:36.433474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5adf9-0c8c-44a5-81be-ea8f62a13b27 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.433748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5adf9-0c8c-44a5-81be-ea8f62a13b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.434237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e5adf9-0c8c-44a5-81be-ea8f62a13b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.434771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ad620-19f9-4941-acd7-74e3f1b415ee service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:36.434910 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5adf9-0c8c-44a5-81be-ea8f62a13b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:36.437165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e5adf9-0c8c-44a5-81be-ea8f62a13b27 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:36.448597 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1032/2078] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:29:36.491160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ad620-19f9-4941-acd7-74e3f1b415ee service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:36.493163 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1047/2079] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:29:36 2022] POST /identity/v3/projects => generated 367 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:36.496930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bc9ad620-19f9-4941-acd7-74e3f1b415ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:36.497627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5adf9-0c8c-44a5-81be-ea8f62a13b27 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:36.515638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ad620-19f9-4941-acd7-74e3f1b415ee service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:36.572105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:36.595130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e5adf9-0c8c-44a5-81be-ea8f62a13b27 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:36.602161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1b201-e965-4b96-bcf4-0b57fbbae7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:36.603120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b201-e965-4b96-bcf4-0b57fbbae7cd admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.603533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b201-e965-4b96-bcf4-0b57fbbae7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.603916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b201-e965-4b96-bcf4-0b57fbbae7cd admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.605652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1b201-e965-4b96-bcf4-0b57fbbae7cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:36.607988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1b201-e965-4b96-bcf4-0b57fbbae7cd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:36.637827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1048/2080] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:29:36 2022] POST /identity/v3/users => generated 284 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:29:36.644473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1b201-e965-4b96-bcf4-0b57fbbae7cd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:36.669568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1b201-e965-4b96-bcf4-0b57fbbae7cd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:36.677175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b5705-9117-433c-9a18-a5fab089ff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:36.678471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b5705-9117-433c-9a18-a5fab089ff5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.678852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70142a7-88d1-4ecc-9f60-c4942223d5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:36.679908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b5705-9117-433c-9a18-a5fab089ff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.680632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b5705-9117-433c-9a18-a5fab089ff5e admin admin] PATH_INFO: `/v3/users/4018a77d5ea4425daf25f03957d3557d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.680772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70142a7-88d1-4ecc-9f60-c4942223d5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.681301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70142a7-88d1-4ecc-9f60-c4942223d5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.682209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70142a7-88d1-4ecc-9f60-c4942223d5b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.683084 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b5705-9117-433c-9a18-a5fab089ff5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4018a77d5ea4425daf25f03957d3557d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:36.683875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70142a7-88d1-4ecc-9f60-c4942223d5b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:36.685704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18b5705-9117-433c-9a18-a5fab089ff5e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:36.687305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70142a7-88d1-4ecc-9f60-c4942223d5b3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:36.714683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1049/2081] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:36.718687 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70142a7-88d1-4ecc-9f60-c4942223d5b3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:36.741099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70142a7-88d1-4ecc-9f60-c4942223d5b3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:36.766901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b18b5705-9117-433c-9a18-a5fab089ff5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4018a77d5ea4425daf25f03957d3557d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:36.768734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b099aba-f660-4cb9-9fc7-7051b05cd7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:36.769418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b099aba-f660-4cb9-9fc7-7051b05cd7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.769700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b099aba-f660-4cb9-9fc7-7051b05cd7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.769985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b099aba-f660-4cb9-9fc7-7051b05cd7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.770777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b099aba-f660-4cb9-9fc7-7051b05cd7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:36.772417 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b099aba-f660-4cb9-9fc7-7051b05cd7ce admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:36.790124 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1050/2082] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:36 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:36.790372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b18b5705-9117-433c-9a18-a5fab089ff5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4018a77d5ea4425daf25f03957d3557d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:36.791874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b099aba-f660-4cb9-9fc7-7051b05cd7ce admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:36.805683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b18b5705-9117-433c-9a18-a5fab089ff5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4018a77d5ea4425daf25f03957d3557d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:36.809060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b099aba-f660-4cb9-9fc7-7051b05cd7ce admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:36.818486 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94441c4-4235-43d8-8903-2b893e61a217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:36.821597 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94441c4-4235-43d8-8903-2b893e61a217 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.822225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94441c4-4235-43d8-8903-2b893e61a217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.822802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94441c4-4235-43d8-8903-2b893e61a217 admin admin] PATH_INFO: `/v3/projects/b9756cc751db40e7a81730ca77faba8e/users/06d673d6c4474920ae5c37a41dfd40d1/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.839385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b18b5705-9117-433c-9a18-a5fab089ff5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4018a77d5ea4425daf25f03957d3557d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:36.840255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b18b5705-9117-433c-9a18-a5fab089ff5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4018a77d5ea4425daf25f03957d3557d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:36.850811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94441c4-4235-43d8-8903-2b893e61a217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9756cc751db40e7a81730ca77faba8e, user_id=06d673d6c4474920ae5c37a41dfd40d1, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:36.854134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1033/2083] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:29:36 2022] DELETE /identity/v3/users/4018a77d5ea4425daf25f03957d3557d => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:36.859365 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b90f32-88ac-4985-b253-73328ed4db68 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.859922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b90f32-88ac-4985-b253-73328ed4db68 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.860364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b90f32-88ac-4985-b253-73328ed4db68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.867745 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94441c4-4235-43d8-8903-2b893e61a217 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:36.892794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1051/2084] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:36 2022] PUT /identity/v3/projects/b9756cc751db40e7a81730ca77faba8e/users/06d673d6c4474920ae5c37a41dfd40d1/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:36.901261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b2b90f32-88ac-4985-b253-73328ed4db68 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:36.904848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b2b90f32-88ac-4985-b253-73328ed4db68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:36.917660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9032a2-23ac-4fcd-b27d-9a4271ba37f3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:36.921236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9032a2-23ac-4fcd-b27d-9a4271ba37f3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:36.922065 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9032a2-23ac-4fcd-b27d-9a4271ba37f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:36.994288 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-2a9032a2-23ac-4fcd-b27d-9a4271ba37f3 None None] MFA Rules not processed for user `06d673d6c4474920ae5c37a41dfd40d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:36.995381 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1034/2085] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:29:37.008402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b90f32-88ac-4985-b253-73328ed4db68 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:37.008609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2a9032a2-23ac-4fcd-b27d-9a4271ba37f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:37.013658 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b2b90f32-88ac-4985-b253-73328ed4db68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:37.123810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b90f32-88ac-4985-b253-73328ed4db68 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:37.124676 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1052/2086] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:37.131096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003b162-0335-4343-9316-f7bc632311c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:37.131803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003b162-0335-4343-9316-f7bc632311c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:37.132453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003b162-0335-4343-9316-f7bc632311c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:37.132959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9032a2-23ac-4fcd-b27d-9a4271ba37f3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:37.133098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2003b162-0335-4343-9316-f7bc632311c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:37.140320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-2a9032a2-23ac-4fcd-b27d-9a4271ba37f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:37.143133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:37.154918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003b162-0335-4343-9316-f7bc632311c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:37.159270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2003b162-0335-4343-9316-f7bc632311c2 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:37.183660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1035/2087] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:29:37.195893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9032a2-23ac-4fcd-b27d-9a4271ba37f3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:37.205482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf4937-6b20-46e1-b766-fea05f5da1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:37.207002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf4937-6b20-46e1-b766-fea05f5da1fe admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:37.211696 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf4937-6b20-46e1-b766-fea05f5da1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:37.212277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf4937-6b20-46e1-b766-fea05f5da1fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:37.215730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf4937-6b20-46e1-b766-fea05f5da1fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:37.219236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cf4937-6b20-46e1-b766-fea05f5da1fe admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:37.238221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:37.280110 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1053/2088] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:29:37 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:37.284081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003b162-0335-4343-9316-f7bc632311c2 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:37.303311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2003b162-0335-4343-9316-f7bc632311c2 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:37.528997 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15caadbc-724a-4489-a14c-52991a696288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:37.529837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caadbc-724a-4489-a14c-52991a696288 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:37.530484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caadbc-724a-4489-a14c-52991a696288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:37.531059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15caadbc-724a-4489-a14c-52991a696288 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:37.532950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15caadbc-724a-4489-a14c-52991a696288 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:37.551033 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15caadbc-724a-4489-a14c-52991a696288 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:37.606770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1036/2089] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:29:37 2022] POST /identity/v3/users => generated 291 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:37.607046 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf4937-6b20-46e1-b766-fea05f5da1fe admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:37.625493 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cf4937-6b20-46e1-b766-fea05f5da1fe admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:37.631852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac50df0-505b-41ad-a315-c48443f0ccf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:37.634952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac50df0-505b-41ad-a315-c48443f0ccf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:37.635300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac50df0-505b-41ad-a315-c48443f0ccf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:37.635768 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac50df0-505b-41ad-a315-c48443f0ccf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:37.636804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac50df0-505b-41ad-a315-c48443f0ccf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:37.640474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac50df0-505b-41ad-a315-c48443f0ccf0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:37.665569 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15caadbc-724a-4489-a14c-52991a696288 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:37.666302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1054/2090] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:37.682342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15caadbc-724a-4489-a14c-52991a696288 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:37.693755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d618136-1186-4ef7-b851-ed93f08f92d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:37.695610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d618136-1186-4ef7-b851-ed93f08f92d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:37.695884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d618136-1186-4ef7-b851-ed93f08f92d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:37.696348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d618136-1186-4ef7-b851-ed93f08f92d6 admin admin] PATH_INFO: `/v3/projects/981e518bc72448f2805b726f9fa31137/users/6c279f95b90542f5a8b7a864812d2278/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:37.727475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d618136-1186-4ef7-b851-ed93f08f92d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=981e518bc72448f2805b726f9fa31137, user_id=6c279f95b90542f5a8b7a864812d2278, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:37.731636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d618136-1186-4ef7-b851-ed93f08f92d6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:37.754834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1037/2091] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:37 2022] PUT /identity/v3/projects/981e518bc72448f2805b726f9fa31137/users/6c279f95b90542f5a8b7a864812d2278/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:29:37.761658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac50df0-505b-41ad-a315-c48443f0ccf0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:37.776401 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac50df0-505b-41ad-a315-c48443f0ccf0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:37.895800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a09bb-fe99-458f-9635-0bd51ae85091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:37.899967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a09bb-fe99-458f-9635-0bd51ae85091 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:37.900896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a09bb-fe99-458f-9635-0bd51ae85091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:37.901634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a09bb-fe99-458f-9635-0bd51ae85091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:37.904157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a09bb-fe99-458f-9635-0bd51ae85091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:37.908601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a09bb-fe99-458f-9635-0bd51ae85091 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:37.932804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1055/2092] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:37.942919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d618136-1186-4ef7-b851-ed93f08f92d6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:37.959644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a09bb-fe99-458f-9635-0bd51ae85091 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:37.963820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-704a09bb-fe99-458f-9635-0bd51ae85091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:37.974822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d618136-1186-4ef7-b851-ed93f08f92d6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:37.986445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59da4cac-115f-46fe-9e1a-02eb62206221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:37.987427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da4cac-115f-46fe-9e1a-02eb62206221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:37.987719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da4cac-115f-46fe-9e1a-02eb62206221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:37.988004 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59da4cac-115f-46fe-9e1a-02eb62206221 admin admin] PATH_INFO: `/v3/projects/981e518bc72448f2805b726f9fa31137/users/6c279f95b90542f5a8b7a864812d2278/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:37.990747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a09bb-fe99-458f-9635-0bd51ae85091 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:37.996282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da4cac-115f-46fe-9e1a-02eb62206221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=981e518bc72448f2805b726f9fa31137, user_id=6c279f95b90542f5a8b7a864812d2278, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:37.998262 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:37.999086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59da4cac-115f-46fe-9e1a-02eb62206221 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:38.001026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.001905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.002431 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] PATH_INFO: `/v3/projects/df0009d53ec14eb48ad8dbe662d5e1d1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.004601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0009d53ec14eb48ad8dbe662d5e1d1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:38.013420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:38.036322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1038/2093] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:37 2022] PUT /identity/v3/projects/981e518bc72448f2805b726f9fa31137/users/6c279f95b90542f5a8b7a864812d2278/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:38.051210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f11c4-b398-4f1d-b8c5-96a7ce15c808 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.051526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f11c4-b398-4f1d-b8c5-96a7ce15c808 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.052876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f11c4-b398-4f1d-b8c5-96a7ce15c808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.137039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8d2f11c4-b398-4f1d-b8c5-96a7ce15c808 None None] MFA Rules not processed for user `6c279f95b90542f5a8b7a864812d2278`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:38.137284 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0009d53ec14eb48ad8dbe662d5e1d1', 'request_id': 'req-cb74804d-40c5-405b-bfa8-7f695471c6aa'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:38.139836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8d2f11c4-b398-4f1d-b8c5-96a7ce15c808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:38.146847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0009d53ec14eb48ad8dbe662d5e1d1', 'request_id': 'req-cb74804d-40c5-405b-bfa8-7f695471c6aa'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:38.160501 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0009d53ec14eb48ad8dbe662d5e1d1', 'request_id': 'req-cb74804d-40c5-405b-bfa8-7f695471c6aa'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:38.181597 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] The token cache is being invalidate because project df0009d53ec14eb48ad8dbe662d5e1d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:29:38.182493 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:38.183665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cb74804d-40c5-405b-bfa8-7f695471c6aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:38.186686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1056/2094] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:29:37 2022] DELETE /identity/v3/projects/df0009d53ec14eb48ad8dbe662d5e1d1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:29:38.193064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd079da5-0905-4b09-9508-65e16273c983 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.193288 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd079da5-0905-4b09-9508-65e16273c983 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.193740 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd079da5-0905-4b09-9508-65e16273c983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.231497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-cd079da5-0905-4b09-9508-65e16273c983 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:38.233566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cd079da5-0905-4b09-9508-65e16273c983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:38.283696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1039/2095] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:38 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:38.296428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f11c4-b398-4f1d-b8c5-96a7ce15c808 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:38.302055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8d2f11c4-b398-4f1d-b8c5-96a7ce15c808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:38.314375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:38.330151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1057/2096] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:38.343678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207b634-779f-4d9e-8c33-f06026e7fb8a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.344146 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207b634-779f-4d9e-8c33-f06026e7fb8a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.345392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207b634-779f-4d9e-8c33-f06026e7fb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.375346 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e207b634-779f-4d9e-8c33-f06026e7fb8a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:38.377438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e207b634-779f-4d9e-8c33-f06026e7fb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:38.386005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:38.406538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1058/2097] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:38.415921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207b634-779f-4d9e-8c33-f06026e7fb8a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:38.428461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207b634-779f-4d9e-8c33-f06026e7fb8a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:38.434988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f11c4-b398-4f1d-b8c5-96a7ce15c808 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:38.436550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed0291-60d1-4181-9e39-0116b0ef2018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:38.437972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed0291-60d1-4181-9e39-0116b0ef2018 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.438607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed0291-60d1-4181-9e39-0116b0ef2018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.439113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed0291-60d1-4181-9e39-0116b0ef2018 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.439601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f1106-139d-4cf6-8716-7253326d6a2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:38.440284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f1106-139d-4cf6-8716-7253326d6a2d service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.440576 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f1106-139d-4cf6-8716-7253326d6a2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.440747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed0291-60d1-4181-9e39-0116b0ef2018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:29:38.440933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f1106-139d-4cf6-8716-7253326d6a2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.441234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed0291-60d1-4181-9e39-0116b0ef2018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:38.446244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed0291-60d1-4181-9e39-0116b0ef2018 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:38.453422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f1106-139d-4cf6-8716-7253326d6a2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:38.454949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f1106-139d-4cf6-8716-7253326d6a2d service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:38.466818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1040/2098] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:38 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:38.478349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1059/2099] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:29:38.479679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f1106-139d-4cf6-8716-7253326d6a2d service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:38.493337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f1106-139d-4cf6-8716-7253326d6a2d service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:38.503129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c817e-a4b5-4819-8980-ea62eec1af0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:38.503908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c817e-a4b5-4819-8980-ea62eec1af0c admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.504284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c817e-a4b5-4819-8980-ea62eec1af0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.504638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c817e-a4b5-4819-8980-ea62eec1af0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.505692 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c817e-a4b5-4819-8980-ea62eec1af0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:38.508914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c817e-a4b5-4819-8980-ea62eec1af0c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:38.562632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1041/2100] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:29:38 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:38.567043 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed0291-60d1-4181-9e39-0116b0ef2018 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:38.582409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed0291-60d1-4181-9e39-0116b0ef2018 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:38.672416 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc9cb7-432e-485d-9c15-dc26444e1b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:38.673387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc9cb7-432e-485d-9c15-dc26444e1b04 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.673779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc9cb7-432e-485d-9c15-dc26444e1b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.674290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc9cb7-432e-485d-9c15-dc26444e1b04 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.675230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc9cb7-432e-485d-9c15-dc26444e1b04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:38.677840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc9cb7-432e-485d-9c15-dc26444e1b04 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:38.708236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1060/2101] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:29:38 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:38.712965 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c817e-a4b5-4819-8980-ea62eec1af0c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:38.723970 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c817e-a4b5-4819-8980-ea62eec1af0c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:38.729888 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6eb0c-9586-40e4-ae2c-96434bddeed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:38.730783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6eb0c-9586-40e4-ae2c-96434bddeed2 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.732216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6eb0c-9586-40e4-ae2c-96434bddeed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.732216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6eb0c-9586-40e4-ae2c-96434bddeed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.732216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6eb0c-9586-40e4-ae2c-96434bddeed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:38.733875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6eb0c-9586-40e4-ae2c-96434bddeed2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:38.751594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1042/2102] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:38.756532 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc9cb7-432e-485d-9c15-dc26444e1b04 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:38.766691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc9cb7-432e-485d-9c15-dc26444e1b04 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:38.772403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f971e0-441f-4249-9e54-8a07038f9532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:38.773236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f971e0-441f-4249-9e54-8a07038f9532 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.773583 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f971e0-441f-4249-9e54-8a07038f9532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.774223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f971e0-441f-4249-9e54-8a07038f9532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.774935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f971e0-441f-4249-9e54-8a07038f9532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:38.776873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f971e0-441f-4249-9e54-8a07038f9532 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:38.794710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1061/2103] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:29:38.800362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6eb0c-9586-40e4-ae2c-96434bddeed2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:38.818128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6eb0c-9586-40e4-ae2c-96434bddeed2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:38.825740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985e1e4-52e8-4ce0-b2d5-60e92cd67252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:38.827226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985e1e4-52e8-4ce0-b2d5-60e92cd67252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.827561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985e1e4-52e8-4ce0-b2d5-60e92cd67252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.827885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985e1e4-52e8-4ce0-b2d5-60e92cd67252 admin admin] PATH_INFO: `/v3/projects/fcfc8ebb18c54de29556d0b93593d676/users/19024dea6ca94745b38158beb08f29fc/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.855525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985e1e4-52e8-4ce0-b2d5-60e92cd67252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcfc8ebb18c54de29556d0b93593d676, user_id=19024dea6ca94745b38158beb08f29fc, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:38.861235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985e1e4-52e8-4ce0-b2d5-60e92cd67252 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:38.881419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1043/2104] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:38 2022] PUT /identity/v3/projects/fcfc8ebb18c54de29556d0b93593d676/users/19024dea6ca94745b38158beb08f29fc/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:38.888226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2fb30-e245-439a-901f-eb1cd419a10a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:38.888710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2fb30-e245-439a-901f-eb1cd419a10a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:38.889164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2fb30-e245-439a-901f-eb1cd419a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:38.910410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b8a2fb30-e245-439a-901f-eb1cd419a10a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:38.912059 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b8a2fb30-e245-439a-901f-eb1cd419a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:38.995762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1062/2105] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:39.008939 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985e1e4-52e8-4ce0-b2d5-60e92cd67252 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:39.013047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3985e1e4-52e8-4ce0-b2d5-60e92cd67252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:39.090544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985e1e4-52e8-4ce0-b2d5-60e92cd67252 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:39.096201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435aa2c-69b5-413a-8ff6-c479b3f1b3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:39.097262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435aa2c-69b5-413a-8ff6-c479b3f1b3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:39.097733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435aa2c-69b5-413a-8ff6-c479b3f1b3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:39.098187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435aa2c-69b5-413a-8ff6-c479b3f1b3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:39.111759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435aa2c-69b5-413a-8ff6-c479b3f1b3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:39.113231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435aa2c-69b5-413a-8ff6-c479b3f1b3c0 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:39.127184 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1044/2106] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:39.489313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cde869-a5cf-46c5-a7d2-a816418959b0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:39.489778 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cde869-a5cf-46c5-a7d2-a816418959b0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:39.490008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cde869-a5cf-46c5-a7d2-a816418959b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:39.519439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-69cde869-a5cf-46c5-a7d2-a816418959b0 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:39.520427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-69cde869-a5cf-46c5-a7d2-a816418959b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:39.599523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1063/2107] 10.222.0.64 () {58 vars in 967 bytes} [Wed Jul 20 08:29:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:29:39.607238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435aa2c-69b5-413a-8ff6-c479b3f1b3c0 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:39.609749 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1435aa2c-69b5-413a-8ff6-c479b3f1b3c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:39.773789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435aa2c-69b5-413a-8ff6-c479b3f1b3c0 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:39.779959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15973f3c-43ed-4b2e-a4c2-7682c6eba37f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:39.781023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15973f3c-43ed-4b2e-a4c2-7682c6eba37f service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:39.781428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15973f3c-43ed-4b2e-a4c2-7682c6eba37f service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:39.781987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15973f3c-43ed-4b2e-a4c2-7682c6eba37f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:39.796771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15973f3c-43ed-4b2e-a4c2-7682c6eba37f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:39.799871 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15973f3c-43ed-4b2e-a4c2-7682c6eba37f service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:39.813521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1045/2108] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:29:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:29:40.377150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cde869-a5cf-46c5-a7d2-a816418959b0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:40.380150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-69cde869-a5cf-46c5-a7d2-a816418959b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:40.393952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:40.405933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cde869-a5cf-46c5-a7d2-a816418959b0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:40.416759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:40.419168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:40.420240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:40.420667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] PATH_INFO: `/v3/users/06d673d6c4474920ae5c37a41dfd40d1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:40.422107 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d673d6c4474920ae5c37a41dfd40d1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:40.424384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:40.477363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d673d6c4474920ae5c37a41dfd40d1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:40.492546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d673d6c4474920ae5c37a41dfd40d1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:40.493281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d673d6c4474920ae5c37a41dfd40d1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:40.502032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d673d6c4474920ae5c37a41dfd40d1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:40.510714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d673d6c4474920ae5c37a41dfd40d1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:40.519559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1064/2109] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:29:40 2022] DELETE /identity/v3/users/06d673d6c4474920ae5c37a41dfd40d1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:40.526035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9786c-bdbc-48a2-8cc4-f5208d885716 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:40.527877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9786c-bdbc-48a2-8cc4-f5208d885716 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:40.528401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9786c-bdbc-48a2-8cc4-f5208d885716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:40.556587 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-96b9786c-bdbc-48a2-8cc4-f5208d885716 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:40.559198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-96b9786c-bdbc-48a2-8cc4-f5208d885716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:40.566012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:40.663136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1046/2110] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:40.679873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:40.681638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:40.776517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58138c-bebc-48c4-8fdc-03c488381969 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:40.784449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc7f65-5de3-465b-8a20-0cbe255c51be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:40.785100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc7f65-5de3-465b-8a20-0cbe255c51be service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:40.785380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc7f65-5de3-465b-8a20-0cbe255c51be service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:40.785817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc7f65-5de3-465b-8a20-0cbe255c51be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:40.803555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc7f65-5de3-465b-8a20-0cbe255c51be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:40.806080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc7f65-5de3-465b-8a20-0cbe255c51be service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:40.819042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1065/2111] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:41.387190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9786c-bdbc-48a2-8cc4-f5208d885716 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:41.394498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-96b9786c-bdbc-48a2-8cc4-f5208d885716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:41.415861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9786c-bdbc-48a2-8cc4-f5208d885716 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:41.422769 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:41.423230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:41.423637 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:41.424034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] PATH_INFO: `/v3/projects/b9756cc751db40e7a81730ca77faba8e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:41.425473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9756cc751db40e7a81730ca77faba8e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:41.427710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:41.499100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9756cc751db40e7a81730ca77faba8e', 'request_id': 'req-e1b741f2-f44f-4595-8b05-45b4fad9312f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:41.510032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9756cc751db40e7a81730ca77faba8e', 'request_id': 'req-e1b741f2-f44f-4595-8b05-45b4fad9312f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:41.518189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9756cc751db40e7a81730ca77faba8e', 'request_id': 'req-e1b741f2-f44f-4595-8b05-45b4fad9312f'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:41.536012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] The token cache is being invalidate because project b9756cc751db40e7a81730ca77faba8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:29:41.537047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:41.538059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:41.541713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1047/2112] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:29:41 2022] DELETE /identity/v3/projects/b9756cc751db40e7a81730ca77faba8e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:29:41.547768 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc7f65-5de3-465b-8a20-0cbe255c51be service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:41.559726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc7f65-5de3-465b-8a20-0cbe255c51be service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:41.632726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:41.634003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:41.634450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:41.634928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] PATH_INFO: `/v3/users/6c279f95b90542f5a8b7a864812d2278` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:41.636381 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c279f95b90542f5a8b7a864812d2278)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:41.638521 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:41.672562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:41.725023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c279f95b90542f5a8b7a864812d2278'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:41.747589 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c279f95b90542f5a8b7a864812d2278'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:41.748421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c279f95b90542f5a8b7a864812d2278'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:41.759327 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c279f95b90542f5a8b7a864812d2278'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:41.771526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c279f95b90542f5a8b7a864812d2278'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:41.781310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1066/2113] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:29:41 2022] DELETE /identity/v3/users/6c279f95b90542f5a8b7a864812d2278 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:42.286559 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:42.289686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:42.315612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b741f2-f44f-4595-8b05-45b4fad9312f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:42.421258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca4763-8035-4822-b645-7b50876c7d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:42.424789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca4763-8035-4822-b645-7b50876c7d24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:42.424789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca4763-8035-4822-b645-7b50876c7d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:42.424789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca4763-8035-4822-b645-7b50876c7d24 admin admin] PATH_INFO: `/v3/users/d88c5f05aab0458d8dc0a285bce33f28` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:42.425854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca4763-8035-4822-b645-7b50876c7d24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d88c5f05aab0458d8dc0a285bce33f28)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:42.429904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca4763-8035-4822-b645-7b50876c7d24 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:42.516862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:42.524079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-eeca4763-8035-4822-b645-7b50876c7d24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd88c5f05aab0458d8dc0a285bce33f28'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:42.542137 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-eeca4763-8035-4822-b645-7b50876c7d24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd88c5f05aab0458d8dc0a285bce33f28'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:42.561089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:42.562825 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:42.567930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-eeca4763-8035-4822-b645-7b50876c7d24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd88c5f05aab0458d8dc0a285bce33f28'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:42.579130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed974e3c-17e9-49e8-aa76-6d59a89c94f5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:42.584522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:42.586309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:42.586509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-eeca4763-8035-4822-b645-7b50876c7d24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd88c5f05aab0458d8dc0a285bce33f28'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:42.587105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:42.587539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] PATH_INFO: `/v3/projects/981e518bc72448f2805b726f9fa31137` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:42.588069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-eeca4763-8035-4822-b645-7b50876c7d24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd88c5f05aab0458d8dc0a285bce33f28'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:42.589813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] RBAC: Authorizing `identity:delete_project(project_id=981e518bc72448f2805b726f9fa31137)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:42.593995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:42.607503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1048/2114] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:29:42 2022] DELETE /identity/v3/users/d88c5f05aab0458d8dc0a285bce33f28 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:42.660508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '981e518bc72448f2805b726f9fa31137', 'request_id': 'req-ec63d990-24f5-42ab-9680-60b87ed7addc'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:42.670945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '981e518bc72448f2805b726f9fa31137', 'request_id': 'req-ec63d990-24f5-42ab-9680-60b87ed7addc'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:42.678161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '981e518bc72448f2805b726f9fa31137', 'request_id': 'req-ec63d990-24f5-42ab-9680-60b87ed7addc'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:42.689627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] The token cache is being invalidate because project 981e518bc72448f2805b726f9fa31137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:29:42.689914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:42.690574 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ec63d990-24f5-42ab-9680-60b87ed7addc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:42.692409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1067/2115] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:29:42 2022] DELETE /identity/v3/projects/981e518bc72448f2805b726f9fa31137 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:29:42.697897 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78c96-e9cb-48d3-8450-9f1a76508920 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:42.698060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78c96-e9cb-48d3-8450-9f1a76508920 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:42.698257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b78c96-e9cb-48d3-8450-9f1a76508920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:42.718215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-75b78c96-e9cb-48d3-8450-9f1a76508920 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:42.719500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-75b78c96-e9cb-48d3-8450-9f1a76508920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:42.784457 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1049/2116] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:42.792782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67aebd-27fc-4495-9f60-2cd6fad42782 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:42.793077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67aebd-27fc-4495-9f60-2cd6fad42782 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:42.793364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67aebd-27fc-4495-9f60-2cd6fad42782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:42.816510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3b67aebd-27fc-4495-9f60-2cd6fad42782 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:42.817757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3b67aebd-27fc-4495-9f60-2cd6fad42782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:42.827862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1068/2117] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:42.831911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b78c96-e9cb-48d3-8450-9f1a76508920 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:42.844880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b78c96-e9cb-48d3-8450-9f1a76508920 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:42.853624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6c100-089c-4d57-86d3-2e9a447cdd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:42.854466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6c100-089c-4d57-86d3-2e9a447cdd67 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:42.854859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6c100-089c-4d57-86d3-2e9a447cdd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:42.855219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6c100-089c-4d57-86d3-2e9a447cdd67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:42.856035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6c100-089c-4d57-86d3-2e9a447cdd67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:29:42.856430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6c100-089c-4d57-86d3-2e9a447cdd67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:42.858173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc6c100-089c-4d57-86d3-2e9a447cdd67 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:42.873232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1050/2118] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:29:42.878841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67aebd-27fc-4495-9f60-2cd6fad42782 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:42.890168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67aebd-27fc-4495-9f60-2cd6fad42782 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:42.897778 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33e387-3440-4e47-b856-a603582bdbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:42.899077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33e387-3440-4e47-b856-a603582bdbeb admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:42.899937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33e387-3440-4e47-b856-a603582bdbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:42.900228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33e387-3440-4e47-b856-a603582bdbeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:42.901246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33e387-3440-4e47-b856-a603582bdbeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:42.903623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33e387-3440-4e47-b856-a603582bdbeb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:42.952255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1069/2119] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:29:42 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:42.957068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6c100-089c-4d57-86d3-2e9a447cdd67 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:42.957688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33e387-3440-4e47-b856-a603582bdbeb admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:42.961533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7bc6c100-089c-4d57-86d3-2e9a447cdd67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:43.018254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6c100-089c-4d57-86d3-2e9a447cdd67 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.051497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:43.069452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33e387-3440-4e47-b856-a603582bdbeb admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.078473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f954a4-3409-47cf-bae7-cc4a69cca8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.079341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f954a4-3409-47cf-bae7-cc4a69cca8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.079746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f954a4-3409-47cf-bae7-cc4a69cca8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.079892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f954a4-3409-47cf-bae7-cc4a69cca8d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.080663 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f954a4-3409-47cf-bae7-cc4a69cca8d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.082747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f954a4-3409-47cf-bae7-cc4a69cca8d5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.113215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1070/2120] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:29:42 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:43.115207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f954a4-3409-47cf-bae7-cc4a69cca8d5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:43.134047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f954a4-3409-47cf-bae7-cc4a69cca8d5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.139986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3ac15-245c-4da4-97f0-9eb905bc485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.140220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.140655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3ac15-245c-4da4-97f0-9eb905bc485f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.140852 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.140965 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3ac15-245c-4da4-97f0-9eb905bc485f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.141146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.141257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3ac15-245c-4da4-97f0-9eb905bc485f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.141428 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] PATH_INFO: `/v3/projects/02622d333fa24a0bbd1ef6f59276ba77` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.142478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3ac15-245c-4da4-97f0-9eb905bc485f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.143868 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02622d333fa24a0bbd1ef6f59276ba77)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.144172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3ac15-245c-4da4-97f0-9eb905bc485f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.147031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.158216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1071/2121] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:43.161840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3ac15-245c-4da4-97f0-9eb905bc485f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:43.173733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3ac15-245c-4da4-97f0-9eb905bc485f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.180356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a563e-caac-4183-b8f3-6e6a77a3376f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.181156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a563e-caac-4183-b8f3-6e6a77a3376f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.181527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a563e-caac-4183-b8f3-6e6a77a3376f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.181867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a563e-caac-4183-b8f3-6e6a77a3376f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.182842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a563e-caac-4183-b8f3-6e6a77a3376f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.184484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a563e-caac-4183-b8f3-6e6a77a3376f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.200857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1072/2122] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:29:43.201783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a563e-caac-4183-b8f3-6e6a77a3376f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:43.216381 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:43.216933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a563e-caac-4183-b8f3-6e6a77a3376f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.229536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02622d333fa24a0bbd1ef6f59276ba77', 'request_id': 'req-defea45f-d643-4448-8b5f-17d190cb3da7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:43.236191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02622d333fa24a0bbd1ef6f59276ba77', 'request_id': 'req-defea45f-d643-4448-8b5f-17d190cb3da7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:43.244654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02622d333fa24a0bbd1ef6f59276ba77', 'request_id': 'req-defea45f-d643-4448-8b5f-17d190cb3da7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:43.265069 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] The token cache is being invalidate because project 02622d333fa24a0bbd1ef6f59276ba77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:29:43.265672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:43.268248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-defea45f-d643-4448-8b5f-17d190cb3da7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:43.270999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1051/2123] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:29:42 2022] DELETE /identity/v3/projects/02622d333fa24a0bbd1ef6f59276ba77 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:43.276202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ea3de-b3c9-4316-b744-65c44448dc18 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.276715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ea3de-b3c9-4316-b744-65c44448dc18 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.277303 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773ea3de-b3c9-4316-b744-65c44448dc18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.312830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-773ea3de-b3c9-4316-b744-65c44448dc18 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:43.314236 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-773ea3de-b3c9-4316-b744-65c44448dc18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:43.316216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cce173-658c-4b09-a751-4f066d37219d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.318809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cce173-658c-4b09-a751-4f066d37219d admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.319693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cce173-658c-4b09-a751-4f066d37219d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.319970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cce173-658c-4b09-a751-4f066d37219d admin admin] PATH_INFO: `/v3/projects/bd673e2040d54e1da2368a257d534e32/users/97920a737afa4cb080089abe717c6c05/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.324659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1052/2124] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:29:43.339123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707539d-5cfe-4868-8904-61ee8f8ddcea None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.339433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707539d-5cfe-4868-8904-61ee8f8ddcea None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.342131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707539d-5cfe-4868-8904-61ee8f8ddcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.365067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cce173-658c-4b09-a751-4f066d37219d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd673e2040d54e1da2368a257d534e32, user_id=97920a737afa4cb080089abe717c6c05, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.371781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cce173-658c-4b09-a751-4f066d37219d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.372931 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-4707539d-5cfe-4868-8904-61ee8f8ddcea None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:43.374247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4707539d-5cfe-4868-8904-61ee8f8ddcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:43.387928 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1053/2125] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:43.392197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707539d-5cfe-4868-8904-61ee8f8ddcea None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:43.408844 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707539d-5cfe-4868-8904-61ee8f8ddcea None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.414964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1073/2126] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:43 2022] PUT /identity/v3/projects/bd673e2040d54e1da2368a257d534e32/users/97920a737afa4cb080089abe717c6c05/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:43.428664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84777-481f-457f-a99c-c70ccd56d8a7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.430707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84777-481f-457f-a99c-c70ccd56d8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.431250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84777-481f-457f-a99c-c70ccd56d8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.491910 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-fbc84777-481f-457f-a99c-c70ccd56d8a7 None None] MFA Rules not processed for user `97920a737afa4cb080089abe717c6c05`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:43.493846 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fbc84777-481f-457f-a99c-c70ccd56d8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:43.542685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9315a-44f7-4776-9bd9-4a42fc68d51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.543626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9315a-44f7-4776-9bd9-4a42fc68d51b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.543892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9315a-44f7-4776-9bd9-4a42fc68d51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.544473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9315a-44f7-4776-9bd9-4a42fc68d51b admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.545867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9315a-44f7-4776-9bd9-4a42fc68d51b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:29:43.546294 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9315a-44f7-4776-9bd9-4a42fc68d51b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.548260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9315a-44f7-4776-9bd9-4a42fc68d51b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.563579 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1054/2127] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:29:43.568539 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9315a-44f7-4776-9bd9-4a42fc68d51b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:43.586413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9315a-44f7-4776-9bd9-4a42fc68d51b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.602611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63296d71-cafc-4ba7-83ce-8e9cc6148d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.604436 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63296d71-cafc-4ba7-83ce-8e9cc6148d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.604801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63296d71-cafc-4ba7-83ce-8e9cc6148d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.605153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63296d71-cafc-4ba7-83ce-8e9cc6148d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.605966 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63296d71-cafc-4ba7-83ce-8e9cc6148d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.608023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63296d71-cafc-4ba7-83ce-8e9cc6148d24 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.641140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1055/2128] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:29:43 2022] POST /identity/v3/projects => generated 395 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:43.645209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63296d71-cafc-4ba7-83ce-8e9cc6148d24 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:43.649385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1074/2129] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:43.659158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc84777-481f-457f-a99c-c70ccd56d8a7 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:43.661361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fbc84777-481f-457f-a99c-c70ccd56d8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:43.720550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:43.740238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63296d71-cafc-4ba7-83ce-8e9cc6148d24 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.745817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611c549-309f-439e-a3bb-63374cc1f0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.746507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611c549-309f-439e-a3bb-63374cc1f0ae admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.746829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611c549-309f-439e-a3bb-63374cc1f0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.747709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611c549-309f-439e-a3bb-63374cc1f0ae admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.748613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611c549-309f-439e-a3bb-63374cc1f0ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.751085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc84777-481f-457f-a99c-c70ccd56d8a7 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.751290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611c549-309f-439e-a3bb-63374cc1f0ae admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.787774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1056/2130] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:29:43 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:43.794200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611c549-309f-439e-a3bb-63374cc1f0ae admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:43.808185 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611c549-309f-439e-a3bb-63374cc1f0ae admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.815230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f4245-3285-4bf4-9204-66374b118344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.816065 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f4245-3285-4bf4-9204-66374b118344 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.817086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f4245-3285-4bf4-9204-66374b118344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.817480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f4245-3285-4bf4-9204-66374b118344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.818341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f4245-3285-4bf4-9204-66374b118344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.821187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f4245-3285-4bf4-9204-66374b118344 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.824187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf2ed2-d41a-45f1-8ada-a0e93ee4c41d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.825886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf2ed2-d41a-45f1-8ada-a0e93ee4c41d service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.826543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf2ed2-d41a-45f1-8ada-a0e93ee4c41d service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.827459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf2ed2-d41a-45f1-8ada-a0e93ee4c41d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.838429 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1057/2131] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:43.842550 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f4245-3285-4bf4-9204-66374b118344 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:43.855768 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f4245-3285-4bf4-9204-66374b118344 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.861624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36ccdd-4889-467a-8325-dbae78d0da1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.862491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36ccdd-4889-467a-8325-dbae78d0da1c admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.863167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36ccdd-4889-467a-8325-dbae78d0da1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.863433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36ccdd-4889-467a-8325-dbae78d0da1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.864784 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36ccdd-4889-467a-8325-dbae78d0da1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.866116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36ccdd-4889-467a-8325-dbae78d0da1c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.867978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:43.879541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1058/2132] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:43.885173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36ccdd-4889-467a-8325-dbae78d0da1c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:43.909969 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36ccdd-4889-467a-8325-dbae78d0da1c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:43.920278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79847780-bb97-4442-9350-55e4597a964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:43.921445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79847780-bb97-4442-9350-55e4597a964a admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:43.921881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79847780-bb97-4442-9350-55e4597a964a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:43.922168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79847780-bb97-4442-9350-55e4597a964a admin admin] PATH_INFO: `/v3/projects/52c707b3bd934babbab9b29f1e734702/users/5bf471d2283c4a6eb9b84d88454ba5e9/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:43.938658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf2ed2-d41a-45f1-8ada-a0e93ee4c41d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.939450 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79847780-bb97-4442-9350-55e4597a964a admin admin] RBAC: Authorizing `identity:create_grant(project_id=52c707b3bd934babbab9b29f1e734702, user_id=5bf471d2283c4a6eb9b84d88454ba5e9, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:43.940721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf2ed2-d41a-45f1-8ada-a0e93ee4c41d service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.942843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79847780-bb97-4442-9350-55e4597a964a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:43.959174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1075/2133] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:29:43.965254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1059/2134] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:43 2022] PUT /identity/v3/projects/52c707b3bd934babbab9b29f1e734702/users/5bf471d2283c4a6eb9b84d88454ba5e9/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:43.984335 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf2ed2-d41a-45f1-8ada-a0e93ee4c41d service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:43.985843 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5eaf2ed2-d41a-45f1-8ada-a0e93ee4c41d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:44.007460 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf2ed2-d41a-45f1-8ada-a0e93ee4c41d service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:44.036874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0da65c-5505-47c6-9798-53696d2ff53f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:44.037166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0da65c-5505-47c6-9798-53696d2ff53f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:44.037478 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0da65c-5505-47c6-9798-53696d2ff53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:44.067985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0e0da65c-5505-47c6-9798-53696d2ff53f None None] MFA Rules not processed for user `5bf471d2283c4a6eb9b84d88454ba5e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:44.070094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0e0da65c-5505-47c6-9798-53696d2ff53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:44.093704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1cc27-5ec3-4f14-82a8-619861f03b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:44.094512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1cc27-5ec3-4f14-82a8-619861f03b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:44.094842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1cc27-5ec3-4f14-82a8-619861f03b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:44.095117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1cc27-5ec3-4f14-82a8-619861f03b2f admin admin] PATH_INFO: `/v3/users/97920a737afa4cb080089abe717c6c05` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:44.096423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1cc27-5ec3-4f14-82a8-619861f03b2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97920a737afa4cb080089abe717c6c05)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:44.098348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1cc27-5ec3-4f14-82a8-619861f03b2f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:44.181696 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:44.185223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0fa1cc27-5ec3-4f14-82a8-619861f03b2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97920a737afa4cb080089abe717c6c05'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:44.214964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0fa1cc27-5ec3-4f14-82a8-619861f03b2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97920a737afa4cb080089abe717c6c05'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:44.218222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0fa1cc27-5ec3-4f14-82a8-619861f03b2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97920a737afa4cb080089abe717c6c05'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:44.230940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1060/2135] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:29:44.234147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0fa1cc27-5ec3-4f14-82a8-619861f03b2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97920a737afa4cb080089abe717c6c05'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:44.243825 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0da65c-5505-47c6-9798-53696d2ff53f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:44.245560 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0e0da65c-5505-47c6-9798-53696d2ff53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:44.246459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0fa1cc27-5ec3-4f14-82a8-619861f03b2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97920a737afa4cb080089abe717c6c05'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:44.260690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1076/2136] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:29:43 2022] DELETE /identity/v3/users/97920a737afa4cb080089abe717c6c05 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:44.265171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c384b-3f1d-48c9-901d-c7fe68489323 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:44.265542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c384b-3f1d-48c9-901d-c7fe68489323 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:44.265896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029c384b-3f1d-48c9-901d-c7fe68489323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:44.288916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-029c384b-3f1d-48c9-901d-c7fe68489323 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:44.290531 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-029c384b-3f1d-48c9-901d-c7fe68489323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:44.345617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0da65c-5505-47c6-9798-53696d2ff53f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:44.378610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1077/2137] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:29:44.390340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c384b-3f1d-48c9-901d-c7fe68489323 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:44.392016 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-029c384b-3f1d-48c9-901d-c7fe68489323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:44.442199 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a022b-a140-43a0-bb6a-0304dbd6ff56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:44.443909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a022b-a140-43a0-bb6a-0304dbd6ff56 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:44.444206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a022b-a140-43a0-bb6a-0304dbd6ff56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:44.445099 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a022b-a140-43a0-bb6a-0304dbd6ff56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:44.452339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-997a022b-a140-43a0-bb6a-0304dbd6ff56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:44.541904 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029c384b-3f1d-48c9-901d-c7fe68489323 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:44.547354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:44.549117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c2c16-fa9d-48f1-a2e5-ce2b25f7a3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:44.549599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c2c16-fa9d-48f1-a2e5-ce2b25f7a3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:44.551538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c2c16-fa9d-48f1-a2e5-ce2b25f7a3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:44.552232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c2c16-fa9d-48f1-a2e5-ce2b25f7a3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:44.570172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c2c16-fa9d-48f1-a2e5-ce2b25f7a3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:44.571956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c2c16-fa9d-48f1-a2e5-ce2b25f7a3c6 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:44.584090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1078/2138] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:44.587307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a23a0d-f18d-4489-a7ed-43c60a605340 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:44.587798 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a23a0d-f18d-4489-a7ed-43c60a605340 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:44.588086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a23a0d-f18d-4489-a7ed-43c60a605340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:44.621788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a022b-a140-43a0-bb6a-0304dbd6ff56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:44.623441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a022b-a140-43a0-bb6a-0304dbd6ff56 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:44.626307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-39a23a0d-f18d-4489-a7ed-43c60a605340 None None] MFA Rules not processed for user `19024dea6ca94745b38158beb08f29fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:44.627631 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-39a23a0d-f18d-4489-a7ed-43c60a605340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:44.644604 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1061/2139] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:44.733447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:44.754090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1079/2140] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:44.756187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a022b-a140-43a0-bb6a-0304dbd6ff56 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:44.759473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-997a022b-a140-43a0-bb6a-0304dbd6ff56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:44.781557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a022b-a140-43a0-bb6a-0304dbd6ff56 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:44.800436 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95eb24-2068-469c-88ff-369bbe9c9cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:44.802696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95eb24-2068-469c-88ff-369bbe9c9cb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:44.803786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95eb24-2068-469c-88ff-369bbe9c9cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:44.804619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95eb24-2068-469c-88ff-369bbe9c9cb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:44.807276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95eb24-2068-469c-88ff-369bbe9c9cb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:44.814616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95eb24-2068-469c-88ff-369bbe9c9cb5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:44.877319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1062/2141] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:29:44 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:44.877856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a23a0d-f18d-4489-a7ed-43c60a605340 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:44.891276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a23a0d-f18d-4489-a7ed-43c60a605340 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:44.978931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce0c7b-c351-48c7-90cf-8d453c5c4244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:44.979658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce0c7b-c351-48c7-90cf-8d453c5c4244 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:44.979957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce0c7b-c351-48c7-90cf-8d453c5c4244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:44.980278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce0c7b-c351-48c7-90cf-8d453c5c4244 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:44.981250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce0c7b-c351-48c7-90cf-8d453c5c4244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:44.983549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce0c7b-c351-48c7-90cf-8d453c5c4244 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:44.997001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95eb24-2068-469c-88ff-369bbe9c9cb5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:44.999197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4d95eb24-2068-469c-88ff-369bbe9c9cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:45.029255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95eb24-2068-469c-88ff-369bbe9c9cb5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:45.038744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1080/2142] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:29:44 2022] POST /identity/v3/users => generated 296 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:29:45.040254 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce0c7b-c351-48c7-90cf-8d453c5c4244 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:45.045353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:45.046641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.047174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.047762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] PATH_INFO: `/v3/projects/bd673e2040d54e1da2368a257d534e32` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.050365 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd673e2040d54e1da2368a257d534e32)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:45.055045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:45.056349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce0c7b-c351-48c7-90cf-8d453c5c4244 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:45.062845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4d8fc-957c-40ef-b1c7-73dd5a162cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:45.063707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d8fc-957c-40ef-b1c7-73dd5a162cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.064138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d8fc-957c-40ef-b1c7-73dd5a162cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.064520 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4d8fc-957c-40ef-b1c7-73dd5a162cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.065469 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4d8fc-957c-40ef-b1c7-73dd5a162cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:45.068042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d4d8fc-957c-40ef-b1c7-73dd5a162cc4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:45.094646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1081/2143] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:29:45.099812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4d8fc-957c-40ef-b1c7-73dd5a162cc4 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:45.123265 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4d8fc-957c-40ef-b1c7-73dd5a162cc4 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:45.135744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f9a5d-17c8-4f3d-afbc-43ebe0101058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:45.136746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9a5d-17c8-4f3d-afbc-43ebe0101058 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.137118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9a5d-17c8-4f3d-afbc-43ebe0101058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.137470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f9a5d-17c8-4f3d-afbc-43ebe0101058 admin admin] PATH_INFO: `/v3/projects/0b42df36a52d4b21b94ccd2d90d1e7a6/users/b1406957b934446f924f32436d597461/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.156068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd673e2040d54e1da2368a257d534e32', 'request_id': 'req-50adb70c-0ca8-4c0e-94e8-dceded936462'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:45.160568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f9a5d-17c8-4f3d-afbc-43ebe0101058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b42df36a52d4b21b94ccd2d90d1e7a6, user_id=b1406957b934446f924f32436d597461, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:45.164978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd673e2040d54e1da2368a257d534e32', 'request_id': 'req-50adb70c-0ca8-4c0e-94e8-dceded936462'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:45.165712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f9a5d-17c8-4f3d-afbc-43ebe0101058 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:45.176552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd673e2040d54e1da2368a257d534e32', 'request_id': 'req-50adb70c-0ca8-4c0e-94e8-dceded936462'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:45.186394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1082/2144] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:45 2022] PUT /identity/v3/projects/0b42df36a52d4b21b94ccd2d90d1e7a6/users/b1406957b934446f924f32436d597461/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:45.190485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f9a5d-17c8-4f3d-afbc-43ebe0101058 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:45.196314 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] The token cache is being invalidate because project bd673e2040d54e1da2368a257d534e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:29:45.197083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:45.199055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-50adb70c-0ca8-4c0e-94e8-dceded936462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:45.203106 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1063/2145] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:29:44 2022] DELETE /identity/v3/projects/bd673e2040d54e1da2368a257d534e32 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:45.211699 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e5a53-2f84-46b7-868c-6792bd8c8dfa None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.212374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e5a53-2f84-46b7-868c-6792bd8c8dfa None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.212810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e5a53-2f84-46b7-868c-6792bd8c8dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.296370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-666e5a53-2f84-46b7-868c-6792bd8c8dfa None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:45.297617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-666e5a53-2f84-46b7-868c-6792bd8c8dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:45.323787 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f9a5d-17c8-4f3d-afbc-43ebe0101058 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:45.344590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c65ef-a191-4c22-9c41-17127aeb1e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:45.345363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c65ef-a191-4c22-9c41-17127aeb1e7d admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.345773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c65ef-a191-4c22-9c41-17127aeb1e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.346128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c65ef-a191-4c22-9c41-17127aeb1e7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.347018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c65ef-a191-4c22-9c41-17127aeb1e7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:45.349574 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c65ef-a191-4c22-9c41-17127aeb1e7d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:45.378716 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1083/2146] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:29:45.381662 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c65ef-a191-4c22-9c41-17127aeb1e7d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:45.383555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c85c65ef-a191-4c22-9c41-17127aeb1e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:45.400366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c65ef-a191-4c22-9c41-17127aeb1e7d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:45.407736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3528d-f579-4fdf-a59f-26970c156848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:45.408963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3528d-f579-4fdf-a59f-26970c156848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.409378 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3528d-f579-4fdf-a59f-26970c156848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.409725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b3528d-f579-4fdf-a59f-26970c156848 admin admin] PATH_INFO: `/v3/projects/0b42df36a52d4b21b94ccd2d90d1e7a6/users/b1406957b934446f924f32436d597461/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.412074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3528d-f579-4fdf-a59f-26970c156848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b42df36a52d4b21b94ccd2d90d1e7a6, user_id=b1406957b934446f924f32436d597461, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:45.415819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b3528d-f579-4fdf-a59f-26970c156848 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:45.426363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1064/2147] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:45.429259 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da4593-6112-476e-81eb-86710daeacfc None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.429808 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da4593-6112-476e-81eb-86710daeacfc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.430397 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da4593-6112-476e-81eb-86710daeacfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.430539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1084/2148] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:45 2022] PUT /identity/v3/projects/0b42df36a52d4b21b94ccd2d90d1e7a6/users/b1406957b934446f924f32436d597461/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:45.457921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-36da4593-6112-476e-81eb-86710daeacfc None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:45.459286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-36da4593-6112-476e-81eb-86710daeacfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:45.500763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:45.553950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1065/2149] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:45.558828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3528d-f579-4fdf-a59f-26970c156848 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:45.572125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b3528d-f579-4fdf-a59f-26970c156848 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:45.577827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51777c-cf32-48a7-af4f-1008dc220010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:45.578045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51777c-cf32-48a7-af4f-1008dc220010 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.578260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51777c-cf32-48a7-af4f-1008dc220010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.578513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f51777c-cf32-48a7-af4f-1008dc220010 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.579292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51777c-cf32-48a7-af4f-1008dc220010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:29:45.579602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51777c-cf32-48a7-af4f-1008dc220010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:45.581278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f51777c-cf32-48a7-af4f-1008dc220010 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:45.598186 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1085/2150] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:29:45.602887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da4593-6112-476e-81eb-86710daeacfc None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:45.612884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da4593-6112-476e-81eb-86710daeacfc None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:45.618246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531277f-b47e-461e-93cd-9afa33d8f0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:45.618970 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531277f-b47e-461e-93cd-9afa33d8f0ec admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.619256 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531277f-b47e-461e-93cd-9afa33d8f0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.619531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e531277f-b47e-461e-93cd-9afa33d8f0ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.620246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531277f-b47e-461e-93cd-9afa33d8f0ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:45.622269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e531277f-b47e-461e-93cd-9afa33d8f0ec admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:45.659975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1066/2151] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:29:45 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:29:45.665116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51777c-cf32-48a7-af4f-1008dc220010 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:45.676954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f51777c-cf32-48a7-af4f-1008dc220010 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:45.694056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:45.758141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bacd6-2a47-45fb-8364-49a09a7cd01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:45.758933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bacd6-2a47-45fb-8364-49a09a7cd01f admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.760070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bacd6-2a47-45fb-8364-49a09a7cd01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.760070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976bacd6-2a47-45fb-8364-49a09a7cd01f admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.760314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bacd6-2a47-45fb-8364-49a09a7cd01f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:45.762463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976bacd6-2a47-45fb-8364-49a09a7cd01f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:45.787599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1086/2152] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:29:45 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:45.792491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531277f-b47e-461e-93cd-9afa33d8f0ec admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:45.806525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e531277f-b47e-461e-93cd-9afa33d8f0ec admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:45.815077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79ca85-6f02-4e9b-9e88-bd7ba638e0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:45.815645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ca85-6f02-4e9b-9e88-bd7ba638e0fb admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.815936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ca85-6f02-4e9b-9e88-bd7ba638e0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.817093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79ca85-6f02-4e9b-9e88-bd7ba638e0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.817946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79ca85-6f02-4e9b-9e88-bd7ba638e0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:45.825326 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79ca85-6f02-4e9b-9e88-bd7ba638e0fb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:45.839486 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1067/2153] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:45.843121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bacd6-2a47-45fb-8364-49a09a7cd01f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:45.858213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976bacd6-2a47-45fb-8364-49a09a7cd01f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:45.866049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93a5ec-1ee6-48cd-b357-cf533b22d62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:45.868414 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93a5ec-1ee6-48cd-b357-cf533b22d62f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.868707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93a5ec-1ee6-48cd-b357-cf533b22d62f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.868982 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93a5ec-1ee6-48cd-b357-cf533b22d62f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.870103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93a5ec-1ee6-48cd-b357-cf533b22d62f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:45.871879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93a5ec-1ee6-48cd-b357-cf533b22d62f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:45.884636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1087/2154] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:45.888378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79ca85-6f02-4e9b-9e88-bd7ba638e0fb admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:45.899972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79ca85-6f02-4e9b-9e88-bd7ba638e0fb admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:45.910908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da5699-52cc-41a7-9ea4-4322fae93bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:45.912031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5699-52cc-41a7-9ea4-4322fae93bc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.912320 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5699-52cc-41a7-9ea4-4322fae93bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.912601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5699-52cc-41a7-9ea4-4322fae93bc7 admin admin] PATH_INFO: `/v3/projects/6a1205a479b84b1dae970d885fa82650/users/72d3bcd64be7490096e87f316eb0d0dd/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:45.931634 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da5699-52cc-41a7-9ea4-4322fae93bc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1205a479b84b1dae970d885fa82650, user_id=72d3bcd64be7490096e87f316eb0d0dd, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:45.934328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da5699-52cc-41a7-9ea4-4322fae93bc7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:45.955145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1068/2155] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:45 2022] PUT /identity/v3/projects/6a1205a479b84b1dae970d885fa82650/users/72d3bcd64be7490096e87f316eb0d0dd/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:29:45.965695 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e685ce-0cda-4541-a044-3d9ae6b60820 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:45.965985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e685ce-0cda-4541-a044-3d9ae6b60820 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:45.966266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e685ce-0cda-4541-a044-3d9ae6b60820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:46.010324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-55e685ce-0cda-4541-a044-3d9ae6b60820 None None] MFA Rules not processed for user `72d3bcd64be7490096e87f316eb0d0dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:46.013237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-55e685ce-0cda-4541-a044-3d9ae6b60820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:46.122643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1088/2156] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:46.127403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da5699-52cc-41a7-9ea4-4322fae93bc7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:46.212502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da5699-52cc-41a7-9ea4-4322fae93bc7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:46.218297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c67f5a-2e61-4e56-8728-4af5a5402bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:46.219130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c67f5a-2e61-4e56-8728-4af5a5402bda admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:46.219526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c67f5a-2e61-4e56-8728-4af5a5402bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:46.219912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c67f5a-2e61-4e56-8728-4af5a5402bda admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:46.220729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c67f5a-2e61-4e56-8728-4af5a5402bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:46.222801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c67f5a-2e61-4e56-8728-4af5a5402bda admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:46.255039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1069/2157] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:29:46 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:46.259734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e685ce-0cda-4541-a044-3d9ae6b60820 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:46.274903 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e685ce-0cda-4541-a044-3d9ae6b60820 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:46.364522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0872185-dd51-40ad-84b5-a24cc40f8615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:46.365228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0872185-dd51-40ad-84b5-a24cc40f8615 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:46.365523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0872185-dd51-40ad-84b5-a24cc40f8615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:46.365809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0872185-dd51-40ad-84b5-a24cc40f8615 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:46.371066 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0872185-dd51-40ad-84b5-a24cc40f8615 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:46.373085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0872185-dd51-40ad-84b5-a24cc40f8615 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:46.411913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1089/2158] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:29:46 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:46.418045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c67f5a-2e61-4e56-8728-4af5a5402bda admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:46.436619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c67f5a-2e61-4e56-8728-4af5a5402bda admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:46.451050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0a367-0e72-412f-aaa0-a30bbbbe646d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:46.451890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0a367-0e72-412f-aaa0-a30bbbbe646d admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:46.452909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0a367-0e72-412f-aaa0-a30bbbbe646d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:46.453273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0a367-0e72-412f-aaa0-a30bbbbe646d admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:46.455135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0a367-0e72-412f-aaa0-a30bbbbe646d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:46.458288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0a367-0e72-412f-aaa0-a30bbbbe646d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:46.486371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1070/2159] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:46.493415 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0872185-dd51-40ad-84b5-a24cc40f8615 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:46.506701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0872185-dd51-40ad-84b5-a24cc40f8615 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:46.512663 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1ee47-389c-49fc-9be9-0666836d0554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:46.513468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ee47-389c-49fc-9be9-0666836d0554 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:46.513828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ee47-389c-49fc-9be9-0666836d0554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:46.514308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1ee47-389c-49fc-9be9-0666836d0554 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:46.515167 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1ee47-389c-49fc-9be9-0666836d0554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:46.516906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1ee47-389c-49fc-9be9-0666836d0554 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:46.539085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1090/2160] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:29:46.543424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0a367-0e72-412f-aaa0-a30bbbbe646d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:46.562316 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0a367-0e72-412f-aaa0-a30bbbbe646d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:46.571126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf823d-c844-422b-a729-46dfa7193ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:46.572048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf823d-c844-422b-a729-46dfa7193ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:46.572340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf823d-c844-422b-a729-46dfa7193ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:46.572624 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf823d-c844-422b-a729-46dfa7193ebc admin admin] PATH_INFO: `/v3/projects/07671256a3ea437b9498e8537a9d1da8/users/ed6a688e55f84ec99e5620da63196428/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:46.599134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf823d-c844-422b-a729-46dfa7193ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=07671256a3ea437b9498e8537a9d1da8, user_id=ed6a688e55f84ec99e5620da63196428, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:46.603988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf823d-c844-422b-a729-46dfa7193ebc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:46.627561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1071/2161] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:46 2022] PUT /identity/v3/projects/07671256a3ea437b9498e8537a9d1da8/users/ed6a688e55f84ec99e5620da63196428/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:46.639963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720a8b-0e9b-4044-8f63-139337da8b91 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:46.640516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720a8b-0e9b-4044-8f63-139337da8b91 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:46.640944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39720a8b-0e9b-4044-8f63-139337da8b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:46.649425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:46.683906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-39720a8b-0e9b-4044-8f63-139337da8b91 None None] MFA Rules not processed for user `ed6a688e55f84ec99e5620da63196428`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:46.685421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-39720a8b-0e9b-4044-8f63-139337da8b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:46.792706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1091/2162] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:46.797398 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf823d-c844-422b-a729-46dfa7193ebc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:46.838659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:46.906153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf823d-c844-422b-a729-46dfa7193ebc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:46.924759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaadd2e-2b2e-448a-a740-2c822768a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:46.925626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaadd2e-2b2e-448a-a740-2c822768a9df admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:46.926049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaadd2e-2b2e-448a-a740-2c822768a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:46.926389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaadd2e-2b2e-448a-a740-2c822768a9df admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:46.927321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaadd2e-2b2e-448a-a740-2c822768a9df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:46.930176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaadd2e-2b2e-448a-a740-2c822768a9df admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:46.973023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1072/2163] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:29:46 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:29:46.974322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39720a8b-0e9b-4044-8f63-139337da8b91 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:46.989201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39720a8b-0e9b-4044-8f63-139337da8b91 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:47.078979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad5b73-c33f-4dd4-9e96-a6a739fc5333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:47.080117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad5b73-c33f-4dd4-9e96-a6a739fc5333 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:47.080405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad5b73-c33f-4dd4-9e96-a6a739fc5333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:47.080681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad5b73-c33f-4dd4-9e96-a6a739fc5333 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:47.089475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad5b73-c33f-4dd4-9e96-a6a739fc5333 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:47.096326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad5b73-c33f-4dd4-9e96-a6a739fc5333 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:47.146454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1092/2164] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:29:46 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:47.152321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaadd2e-2b2e-448a-a740-2c822768a9df admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:47.167082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaadd2e-2b2e-448a-a740-2c822768a9df admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:47.174546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6589e7-434d-485f-b420-741800d74809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:47.175269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6589e7-434d-485f-b420-741800d74809 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:47.175558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6589e7-434d-485f-b420-741800d74809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:47.175839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6589e7-434d-485f-b420-741800d74809 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:47.176775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6589e7-434d-485f-b420-741800d74809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:47.179263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6589e7-434d-485f-b420-741800d74809 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:47.198074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1073/2165] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:47.199079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad5b73-c33f-4dd4-9e96-a6a739fc5333 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:47.212075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad5b73-c33f-4dd4-9e96-a6a739fc5333 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:47.217474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb08ca-f08f-4dd3-964b-51364d886990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:47.217981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb08ca-f08f-4dd3-964b-51364d886990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:47.218267 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb08ca-f08f-4dd3-964b-51364d886990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:47.218546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb08ca-f08f-4dd3-964b-51364d886990 admin admin] PATH_INFO: `/v3/projects/44210f20f39e406bb7203ce6f31bee58/users/5978110919864b148f80f95932be8bb8/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:47.261366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb08ca-f08f-4dd3-964b-51364d886990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44210f20f39e406bb7203ce6f31bee58, user_id=5978110919864b148f80f95932be8bb8, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:47.264660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb08ca-f08f-4dd3-964b-51364d886990 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:47.289800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1093/2166] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:47 2022] PUT /identity/v3/projects/44210f20f39e406bb7203ce6f31bee58/users/5978110919864b148f80f95932be8bb8/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:47.290925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6589e7-434d-485f-b420-741800d74809 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:47.301986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6589e7-434d-485f-b420-741800d74809 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:47.367381 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18df230-9efc-4618-8738-d916f513b1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:47.368226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18df230-9efc-4618-8738-d916f513b1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:47.368609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18df230-9efc-4618-8738-d916f513b1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:47.368983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18df230-9efc-4618-8738-d916f513b1f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:47.369814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18df230-9efc-4618-8738-d916f513b1f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:47.371580 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18df230-9efc-4618-8738-d916f513b1f0 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:47.388228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb08ca-f08f-4dd3-964b-51364d886990 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:47.390430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1074/2167] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:47.401276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb08ca-f08f-4dd3-964b-51364d886990 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:47.409578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa439f9f-59d9-49d1-b03a-8229ffebbf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:47.410858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa439f9f-59d9-49d1-b03a-8229ffebbf3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:47.411158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa439f9f-59d9-49d1-b03a-8229ffebbf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:47.411977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa439f9f-59d9-49d1-b03a-8229ffebbf3d admin admin] PATH_INFO: `/v3/projects/44210f20f39e406bb7203ce6f31bee58/users/5978110919864b148f80f95932be8bb8/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:47.414952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa439f9f-59d9-49d1-b03a-8229ffebbf3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=44210f20f39e406bb7203ce6f31bee58, user_id=5978110919864b148f80f95932be8bb8, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:47.417680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa439f9f-59d9-49d1-b03a-8229ffebbf3d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:47.474633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1094/2168] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:47 2022] PUT /identity/v3/projects/44210f20f39e406bb7203ce6f31bee58/users/5978110919864b148f80f95932be8bb8/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:47.483380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65751dca-f35a-4858-8bba-41f88451cf6f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:47.484099 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65751dca-f35a-4858-8bba-41f88451cf6f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:47.484099 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65751dca-f35a-4858-8bba-41f88451cf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:47.527410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-65751dca-f35a-4858-8bba-41f88451cf6f None None] MFA Rules not processed for user `5978110919864b148f80f95932be8bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:47.528658 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-65751dca-f35a-4858-8bba-41f88451cf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:47.635994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1075/2169] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:47.647763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa439f9f-59d9-49d1-b03a-8229ffebbf3d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:47.650403 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fa439f9f-59d9-49d1-b03a-8229ffebbf3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:47.766700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:47.776659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa439f9f-59d9-49d1-b03a-8229ffebbf3d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:47.783317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03beca-7838-4104-9f1c-7323ef443fb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:47.784164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03beca-7838-4104-9f1c-7323ef443fb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:47.784908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03beca-7838-4104-9f1c-7323ef443fb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:47.785406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03beca-7838-4104-9f1c-7323ef443fb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:47.870856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03beca-7838-4104-9f1c-7323ef443fb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:47.872548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd03beca-7838-4104-9f1c-7323ef443fb8 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:47.886719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1095/2170] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:50.553059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9519e6c-57cc-434b-b4b8-b96e241d9ea6 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:50.553594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9519e6c-57cc-434b-b4b8-b96e241d9ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:50.553594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9519e6c-57cc-434b-b4b8-b96e241d9ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:50.590028 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c9519e6c-57cc-434b-b4b8-b96e241d9ea6 None None] MFA Rules not processed for user `b1406957b934446f924f32436d597461`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:50.591517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c9519e6c-57cc-434b-b4b8-b96e241d9ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:50.603744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:50.685224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1076/2171] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:50.701092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03beca-7838-4104-9f1c-7323ef443fb8 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:50.702870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dd03beca-7838-4104-9f1c-7323ef443fb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:50.782889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03beca-7838-4104-9f1c-7323ef443fb8 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:50.788554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ed5cc-f4a4-44dc-971f-d06aa377233a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:50.789214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed5cc-f4a4-44dc-971f-d06aa377233a service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:50.789503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed5cc-f4a4-44dc-971f-d06aa377233a service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:50.789789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ed5cc-f4a4-44dc-971f-d06aa377233a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:50.791642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e67ed5cc-f4a4-44dc-971f-d06aa377233a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:50.858779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ed5cc-f4a4-44dc-971f-d06aa377233a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:50.860397 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ed5cc-f4a4-44dc-971f-d06aa377233a service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:50.871845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1096/2172] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:55.139999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9519e6c-57cc-434b-b4b8-b96e241d9ea6 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:55.141525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c9519e6c-57cc-434b-b4b8-b96e241d9ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:55.156224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9519e6c-57cc-434b-b4b8-b96e241d9ea6 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:55.159898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2145bb-ecee-41d7-8586-4d8e5925e10a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:55.160570 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2145bb-ecee-41d7-8586-4d8e5925e10a service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:55.160858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2145bb-ecee-41d7-8586-4d8e5925e10a service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:55.161129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2145bb-ecee-41d7-8586-4d8e5925e10a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:55.249659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2145bb-ecee-41d7-8586-4d8e5925e10a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:55.251311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2145bb-ecee-41d7-8586-4d8e5925e10a service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:55.263210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1077/2173] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:55.989729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ed5cc-f4a4-44dc-971f-d06aa377233a service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:55.999552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ed5cc-f4a4-44dc-971f-d06aa377233a service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:56.003147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8dbae1-9dde-4205-8635-0a8d396b5c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:56.003801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8dbae1-9dde-4205-8635-0a8d396b5c05 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:56.004087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8dbae1-9dde-4205-8635-0a8d396b5c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:56.004365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8dbae1-9dde-4205-8635-0a8d396b5c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:56.015003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8dbae1-9dde-4205-8635-0a8d396b5c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:56.016466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8dbae1-9dde-4205-8635-0a8d396b5c05 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:56.027374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1097/2174] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:56.473586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2145bb-ecee-41d7-8586-4d8e5925e10a service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:56.475146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-be2145bb-ecee-41d7-8586-4d8e5925e10a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:56.546863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2145bb-ecee-41d7-8586-4d8e5925e10a service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:56.552448 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:56.553138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:56.553445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:56.554267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] PATH_INFO: `/v3/users/5bf471d2283c4a6eb9b84d88454ba5e9` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:56.555855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf471d2283c4a6eb9b84d88454ba5e9)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:56.557830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:56.609612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf471d2283c4a6eb9b84d88454ba5e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:56.610847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:56.619454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf471d2283c4a6eb9b84d88454ba5e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:56.628490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf471d2283c4a6eb9b84d88454ba5e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:56.642627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf471d2283c4a6eb9b84d88454ba5e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:56.643291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf471d2283c4a6eb9b84d88454ba5e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:56.649642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1078/2175] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:29:56 2022] DELETE /identity/v3/users/5bf471d2283c4a6eb9b84d88454ba5e9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:56.654468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc724-3797-4ebe-86ca-182c33c8805d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:56.654791 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc724-3797-4ebe-86ca-182c33c8805d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:56.655079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdadc724-3797-4ebe-86ca-182c33c8805d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:56.675517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bdadc724-3797-4ebe-86ca-182c33c8805d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:56.676776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bdadc724-3797-4ebe-86ca-182c33c8805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:56.699115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:56.744537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1098/2176] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:56.754511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:56.756262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:56.816789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73d4d5-cf68-48a7-9871-651dcbef3f8f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:56.821019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13ffa3-52d2-495e-9f3b-8b31969e7888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:56.821694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13ffa3-52d2-495e-9f3b-8b31969e7888 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:56.821982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13ffa3-52d2-495e-9f3b-8b31969e7888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:56.822266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13ffa3-52d2-495e-9f3b-8b31969e7888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:56.832288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13ffa3-52d2-495e-9f3b-8b31969e7888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:56.833745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13ffa3-52d2-495e-9f3b-8b31969e7888 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:56.844641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1079/2177] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:29:57.175061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadc724-3797-4ebe-86ca-182c33c8805d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:57.176618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bdadc724-3797-4ebe-86ca-182c33c8805d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:57.193234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdadc724-3797-4ebe-86ca-182c33c8805d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:57.199045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:57.199964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:57.200248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:57.200666 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] PATH_INFO: `/v3/projects/52c707b3bd934babbab9b29f1e734702` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:57.201923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] RBAC: Authorizing `identity:delete_project(project_id=52c707b3bd934babbab9b29f1e734702)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:57.203957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:57.267771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52c707b3bd934babbab9b29f1e734702', 'request_id': 'req-176ca8be-5506-4f7a-af53-a509b8c7843a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:57.276258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52c707b3bd934babbab9b29f1e734702', 'request_id': 'req-176ca8be-5506-4f7a-af53-a509b8c7843a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:57.284274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52c707b3bd934babbab9b29f1e734702', 'request_id': 'req-176ca8be-5506-4f7a-af53-a509b8c7843a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:57.295009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] The token cache is being invalidate because project 52c707b3bd934babbab9b29f1e734702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:29:57.295395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:57.295973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-176ca8be-5506-4f7a-af53-a509b8c7843a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:29:57.297515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1099/2178] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:29:57 2022] DELETE /identity/v3/projects/52c707b3bd934babbab9b29f1e734702 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:57.302654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252a062-b8fa-4455-aa5a-77f8dec8f5ba None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:57.302963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252a062-b8fa-4455-aa5a-77f8dec8f5ba None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:57.303341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252a062-b8fa-4455-aa5a-77f8dec8f5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:57.323783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d252a062-b8fa-4455-aa5a-77f8dec8f5ba None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:57.325081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d252a062-b8fa-4455-aa5a-77f8dec8f5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:57.398766 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1080/2179] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:57.402436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5af3b4-4202-4e31-b824-1ee648a559c3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:57.402764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5af3b4-4202-4e31-b824-1ee648a559c3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:57.403037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5af3b4-4202-4e31-b824-1ee648a559c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:57.423952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3e5af3b4-4202-4e31-b824-1ee648a559c3 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:57.425376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3e5af3b4-4202-4e31-b824-1ee648a559c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:57.438546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1100/2180] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:57.439566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252a062-b8fa-4455-aa5a-77f8dec8f5ba None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:57.451593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252a062-b8fa-4455-aa5a-77f8dec8f5ba None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:57.457237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfff8b-27ab-4315-b00f-08dc970cecd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:57.458291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfff8b-27ab-4315-b00f-08dc970cecd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:57.458674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfff8b-27ab-4315-b00f-08dc970cecd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:57.458954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfff8b-27ab-4315-b00f-08dc970cecd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:57.459701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfff8b-27ab-4315-b00f-08dc970cecd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:29:57.460131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfff8b-27ab-4315-b00f-08dc970cecd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:57.461861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfff8b-27ab-4315-b00f-08dc970cecd0 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:57.476066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1081/2181] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:29:57.480486 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5af3b4-4202-4e31-b824-1ee648a559c3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:57.490439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5af3b4-4202-4e31-b824-1ee648a559c3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:57.495963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b4873-103b-45a0-952d-3a0c794941eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:57.496628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b4873-103b-45a0-952d-3a0c794941eb admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:57.496913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b4873-103b-45a0-952d-3a0c794941eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:57.497215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b4873-103b-45a0-952d-3a0c794941eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:57.497916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b4873-103b-45a0-952d-3a0c794941eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:57.500028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b4873-103b-45a0-952d-3a0c794941eb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:57.526257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1101/2182] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:29:57 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:29:57.530050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfff8b-27ab-4315-b00f-08dc970cecd0 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:57.540700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfff8b-27ab-4315-b00f-08dc970cecd0 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:57.594850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:57.604484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56264762-b2bc-4b45-92de-57b860a631c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:57.605210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56264762-b2bc-4b45-92de-57b860a631c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:57.605499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56264762-b2bc-4b45-92de-57b860a631c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:57.605780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56264762-b2bc-4b45-92de-57b860a631c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:57.606565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56264762-b2bc-4b45-92de-57b860a631c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:57.608575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56264762-b2bc-4b45-92de-57b860a631c8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:57.630459 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1082/2183] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:29:57 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:29:57.634872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b4873-103b-45a0-952d-3a0c794941eb admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:57.645139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b4873-103b-45a0-952d-3a0c794941eb admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:57.649768 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e1aca-10c3-4ba6-925f-77f4bf0b3e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:57.650477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e1aca-10c3-4ba6-925f-77f4bf0b3e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:57.650795 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e1aca-10c3-4ba6-925f-77f4bf0b3e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:57.651071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e1aca-10c3-4ba6-925f-77f4bf0b3e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:57.651804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e1aca-10c3-4ba6-925f-77f4bf0b3e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:57.653422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e1aca-10c3-4ba6-925f-77f4bf0b3e82 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:57.665722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1102/2184] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:57.669708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56264762-b2bc-4b45-92de-57b860a631c8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:57.680932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56264762-b2bc-4b45-92de-57b860a631c8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:57.685546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653538b-825a-46b3-bd4a-3b0a05964342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:57.686204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653538b-825a-46b3-bd4a-3b0a05964342 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:57.686492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653538b-825a-46b3-bd4a-3b0a05964342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:57.686819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653538b-825a-46b3-bd4a-3b0a05964342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:57.687532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653538b-825a-46b3-bd4a-3b0a05964342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:57.689118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653538b-825a-46b3-bd4a-3b0a05964342 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:57.699460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1083/2185] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:29:57.703655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e1aca-10c3-4ba6-925f-77f4bf0b3e82 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:57.714259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e1aca-10c3-4ba6-925f-77f4bf0b3e82 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:57.719099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3aa175-030c-4417-a476-5e584cef51d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:57.719992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aa175-030c-4417-a476-5e584cef51d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:57.720274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aa175-030c-4417-a476-5e584cef51d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:57.720552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3aa175-030c-4417-a476-5e584cef51d8 admin admin] PATH_INFO: `/v3/projects/714941c1e0034a37b0bf0144c1eac958/users/9c3ca706dfed4770a1eef107b28a9233/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:57.736924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3aa175-030c-4417-a476-5e584cef51d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=714941c1e0034a37b0bf0144c1eac958, user_id=9c3ca706dfed4770a1eef107b28a9233, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:57.739829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3aa175-030c-4417-a476-5e584cef51d8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:57.766643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1103/2186] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:29:57 2022] PUT /identity/v3/projects/714941c1e0034a37b0bf0144c1eac958/users/9c3ca706dfed4770a1eef107b28a9233/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:29:57.773313 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bd90a-7af6-4fc7-8d9f-425462db2e4d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:57.773635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bd90a-7af6-4fc7-8d9f-425462db2e4d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:57.773916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bd90a-7af6-4fc7-8d9f-425462db2e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:57.792389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9b1bd90a-7af6-4fc7-8d9f-425462db2e4d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:29:57.793603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9b1bd90a-7af6-4fc7-8d9f-425462db2e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:57.851726 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1084/2187] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:29:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:29:57.861418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3aa175-030c-4417-a476-5e584cef51d8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:29:57.863190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0a3aa175-030c-4417-a476-5e584cef51d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:29:57.914255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:29:57.930179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3aa175-030c-4417-a476-5e584cef51d8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:29:57.933545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736db4e8-3ff4-4b49-a0f1-02c4abba403f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:29:57.934122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db4e8-3ff4-4b49-a0f1-02c4abba403f service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:29:57.934411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db4e8-3ff4-4b49-a0f1-02c4abba403f service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:29:57.934723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db4e8-3ff4-4b49-a0f1-02c4abba403f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:29:57.945568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736db4e8-3ff4-4b49-a0f1-02c4abba403f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:29:57.947339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736db4e8-3ff4-4b49-a0f1-02c4abba403f service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:29:57.960436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1104/2188] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:29:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:30:01.271588 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bd90a-7af6-4fc7-8d9f-425462db2e4d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:01.273582 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9b1bd90a-7af6-4fc7-8d9f-425462db2e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:01.301093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bd90a-7af6-4fc7-8d9f-425462db2e4d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:01.311246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:01.312404 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:01.313433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:01.314376 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] PATH_INFO: `/v3/users/72d3bcd64be7490096e87f316eb0d0dd` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:01.316141 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d3bcd64be7490096e87f316eb0d0dd)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:01.321115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:01.456912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d3bcd64be7490096e87f316eb0d0dd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:01.477186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d3bcd64be7490096e87f316eb0d0dd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:01.512610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d3bcd64be7490096e87f316eb0d0dd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:01.550600 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d3bcd64be7490096e87f316eb0d0dd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:01.550600 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d3bcd64be7490096e87f316eb0d0dd'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:01.567755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1085/2189] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:30:01 2022] DELETE /identity/v3/users/72d3bcd64be7490096e87f316eb0d0dd => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:30:01.572908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793cedc-6502-455b-b10a-d0a497cb3826 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:01.573969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793cedc-6502-455b-b10a-d0a497cb3826 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:01.573969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e793cedc-6502-455b-b10a-d0a497cb3826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:01.603545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e793cedc-6502-455b-b10a-d0a497cb3826 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:01.604862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e793cedc-6502-455b-b10a-d0a497cb3826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:01.701542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1105/2190] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:30:01.715747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:01.718382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:01.799207 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:01.811882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69b6ca-2a47-4386-b6a8-9820e326ed64 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:01.826451 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b4afa-2380-4e8c-ae99-98504cdfc529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:01.827924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b4afa-2380-4e8c-ae99-98504cdfc529 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:01.828219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b4afa-2380-4e8c-ae99-98504cdfc529 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:01.828508 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b4afa-2380-4e8c-ae99-98504cdfc529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:01.848064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b4afa-2380-4e8c-ae99-98504cdfc529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:01.851994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b4afa-2380-4e8c-ae99-98504cdfc529 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:01.867652 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1086/2191] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:30:02.669930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793cedc-6502-455b-b10a-d0a497cb3826 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:02.672368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e793cedc-6502-455b-b10a-d0a497cb3826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:02.718260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e793cedc-6502-455b-b10a-d0a497cb3826 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:02.725364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:02.727751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:02.728062 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:02.728526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] PATH_INFO: `/v3/projects/6a1205a479b84b1dae970d885fa82650` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:02.730481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a1205a479b84b1dae970d885fa82650)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:02.733082 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:02.814313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a1205a479b84b1dae970d885fa82650', 'request_id': 'req-4b1b0d9a-828c-475d-98c0-83671de48bea'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:02.826200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a1205a479b84b1dae970d885fa82650', 'request_id': 'req-4b1b0d9a-828c-475d-98c0-83671de48bea'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:02.836434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a1205a479b84b1dae970d885fa82650', 'request_id': 'req-4b1b0d9a-828c-475d-98c0-83671de48bea'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:02.860509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] The token cache is being invalidate because project 6a1205a479b84b1dae970d885fa82650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:30:02.861026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:02.862146 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:02.865333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1106/2192] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:30:02 2022] DELETE /identity/v3/projects/6a1205a479b84b1dae970d885fa82650 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:30:02.871777 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b4afa-2380-4e8c-ae99-98504cdfc529 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:02.887361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b4afa-2380-4e8c-ae99-98504cdfc529 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:02.971481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e122259-9265-4b12-b0fd-6b4ec390ae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:02.973236 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e122259-9265-4b12-b0fd-6b4ec390ae64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:02.973577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e122259-9265-4b12-b0fd-6b4ec390ae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:02.974143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e122259-9265-4b12-b0fd-6b4ec390ae64 admin admin] PATH_INFO: `/v3/users/ed6a688e55f84ec99e5620da63196428` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:02.978084 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e122259-9265-4b12-b0fd-6b4ec390ae64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed6a688e55f84ec99e5620da63196428)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:02.980883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e122259-9265-4b12-b0fd-6b4ec390ae64 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:03.095638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2e122259-9265-4b12-b0fd-6b4ec390ae64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed6a688e55f84ec99e5620da63196428'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:03.113258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2e122259-9265-4b12-b0fd-6b4ec390ae64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed6a688e55f84ec99e5620da63196428'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:03.132415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2e122259-9265-4b12-b0fd-6b4ec390ae64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed6a688e55f84ec99e5620da63196428'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:03.152274 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2e122259-9265-4b12-b0fd-6b4ec390ae64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed6a688e55f84ec99e5620da63196428'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:03.153283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2e122259-9265-4b12-b0fd-6b4ec390ae64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed6a688e55f84ec99e5620da63196428'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:03.160290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1087/2193] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:30:02 2022] DELETE /identity/v3/users/ed6a688e55f84ec99e5620da63196428 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:30:03.644999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:03.652087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:03.679561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b0d9a-828c-475d-98c0-83671de48bea admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:03.697539 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea9e03-653f-4cbe-bf8f-1c4187fbb675 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:03.698042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea9e03-653f-4cbe-bf8f-1c4187fbb675 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:03.698520 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea9e03-653f-4cbe-bf8f-1c4187fbb675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:03.740839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b9ea9e03-653f-4cbe-bf8f-1c4187fbb675 None None] MFA Rules not processed for user `9c3ca706dfed4770a1eef107b28a9233`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:03.743149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b9ea9e03-653f-4cbe-bf8f-1c4187fbb675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:03.775714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:03.776559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:03.777030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:03.777314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] PATH_INFO: `/v3/projects/07671256a3ea437b9498e8537a9d1da8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:03.778553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07671256a3ea437b9498e8537a9d1da8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:03.780577 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:03.802298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:03.824503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:03.846098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1088/2194] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:30:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:30:03.847610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea9e03-653f-4cbe-bf8f-1c4187fbb675 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:03.850394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b9ea9e03-653f-4cbe-bf8f-1c4187fbb675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:03.875172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea9e03-653f-4cbe-bf8f-1c4187fbb675 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:03.895765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07671256a3ea437b9498e8537a9d1da8', 'request_id': 'req-3bf9f0b1-c05c-4167-a55c-d34448d56961'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:03.908690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07671256a3ea437b9498e8537a9d1da8', 'request_id': 'req-3bf9f0b1-c05c-4167-a55c-d34448d56961'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:03.921743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07671256a3ea437b9498e8537a9d1da8', 'request_id': 'req-3bf9f0b1-c05c-4167-a55c-d34448d56961'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:03.941197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] The token cache is being invalidate because project 07671256a3ea437b9498e8537a9d1da8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:30:03.941556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:03.942176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:03.944171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1107/2195] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:30:03 2022] DELETE /identity/v3/projects/07671256a3ea437b9498e8537a9d1da8 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:30:03.949452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:03.951424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:03.979886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4a95f-ee0e-43b0-b4e1-39ba922c0509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:03.980713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4a95f-ee0e-43b0-b4e1-39ba922c0509 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:03.981114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4a95f-ee0e-43b0-b4e1-39ba922c0509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:03.981487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb4a95f-ee0e-43b0-b4e1-39ba922c0509 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:03.982287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4a95f-ee0e-43b0-b4e1-39ba922c0509 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:03.986054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb4a95f-ee0e-43b0-b4e1-39ba922c0509 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:04.034178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9f0b1-c05c-4167-a55c-d34448d56961 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:04.038978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1089/2196] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:30:03 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:30:04.041456 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4a95f-ee0e-43b0-b4e1-39ba922c0509 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:04.046928 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dcb4a95f-ee0e-43b0-b4e1-39ba922c0509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:04.131511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:04.132815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:04.133215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:04.133594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] PATH_INFO: `/v3/users/5978110919864b148f80f95932be8bb8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:04.135466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5978110919864b148f80f95932be8bb8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:04.137566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:04.157501 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb4a95f-ee0e-43b0-b4e1-39ba922c0509 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:04.162929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec23a-0aea-4747-8ffa-97d9288089d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:04.164216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec23a-0aea-4747-8ffa-97d9288089d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:04.164505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec23a-0aea-4747-8ffa-97d9288089d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:04.164789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ec23a-0aea-4747-8ffa-97d9288089d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:04.165498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ec23a-0aea-4747-8ffa-97d9288089d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:04.167631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ec23a-0aea-4747-8ffa-97d9288089d2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:04.195455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1090/2197] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:30:04 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:30:04.201621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec23a-0aea-4747-8ffa-97d9288089d2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:04.219860 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5978110919864b148f80f95932be8bb8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:04.220140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ec23a-0aea-4747-8ffa-97d9288089d2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:04.224899 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:04.229346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901152d1-311d-42b9-963c-df550bcf5577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:04.230465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901152d1-311d-42b9-963c-df550bcf5577 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:04.230992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901152d1-311d-42b9-963c-df550bcf5577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:04.231468 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901152d1-311d-42b9-963c-df550bcf5577 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:04.232809 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901152d1-311d-42b9-963c-df550bcf5577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:04.236264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901152d1-311d-42b9-963c-df550bcf5577 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:04.247756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5978110919864b148f80f95932be8bb8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:04.249134 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5978110919864b148f80f95932be8bb8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:04.258926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1091/2198] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:30:04.260881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901152d1-311d-42b9-963c-df550bcf5577 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:04.262231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5978110919864b148f80f95932be8bb8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:04.273687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901152d1-311d-42b9-963c-df550bcf5577 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:04.277937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5978110919864b148f80f95932be8bb8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:04.278814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7866f4-e0d8-421b-9a81-3b9887b64d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:04.279721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7866f4-e0d8-421b-9a81-3b9887b64d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:04.280007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7866f4-e0d8-421b-9a81-3b9887b64d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:04.280288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7866f4-e0d8-421b-9a81-3b9887b64d4b admin admin] PATH_INFO: `/v3/projects/c3cf87b84407454091a026745dfe006b/users/50d228e676244ec8b45b978665a7b2da/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:04.289594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1108/2199] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:30:03 2022] DELETE /identity/v3/users/5978110919864b148f80f95932be8bb8 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:30:04.305373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7866f4-e0d8-421b-9a81-3b9887b64d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3cf87b84407454091a026745dfe006b, user_id=50d228e676244ec8b45b978665a7b2da, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:04.309598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7866f4-e0d8-421b-9a81-3b9887b64d4b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:04.325660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1092/2200] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:30:04 2022] PUT /identity/v3/projects/c3cf87b84407454091a026745dfe006b/users/50d228e676244ec8b45b978665a7b2da/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:30:04.331542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:04.333186 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:04.465642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d32076-c9bf-4c08-9388-b097d8bc92fd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:04.559438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fd23e-5a0d-43fd-aa5c-e12e0c003dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:04.560240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fd23e-5a0d-43fd-aa5c-e12e0c003dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:04.560655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fd23e-5a0d-43fd-aa5c-e12e0c003dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:04.561007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28fd23e-5a0d-43fd-aa5c-e12e0c003dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:04.562182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fd23e-5a0d-43fd-aa5c-e12e0c003dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:04.564883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28fd23e-5a0d-43fd-aa5c-e12e0c003dbf admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:04.576855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1109/2201] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:30:04.581548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7866f4-e0d8-421b-9a81-3b9887b64d4b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:04.608566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7866f4-e0d8-421b-9a81-3b9887b64d4b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:04.615065 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bf31e-924c-4cb7-bbd3-953a209dffaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:04.617667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bf31e-924c-4cb7-bbd3-953a209dffaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:04.617667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bf31e-924c-4cb7-bbd3-953a209dffaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:04.617667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bf31e-924c-4cb7-bbd3-953a209dffaf admin admin] PATH_INFO: `/v3/projects/c3cf87b84407454091a026745dfe006b/users/50d228e676244ec8b45b978665a7b2da/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:04.620264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bf31e-924c-4cb7-bbd3-953a209dffaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3cf87b84407454091a026745dfe006b, user_id=50d228e676244ec8b45b978665a7b2da, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:04.623061 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bf31e-924c-4cb7-bbd3-953a209dffaf admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:04.640017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1093/2202] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:30:04 2022] PUT /identity/v3/projects/c3cf87b84407454091a026745dfe006b/users/50d228e676244ec8b45b978665a7b2da/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:30:04.756334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fd23e-5a0d-43fd-aa5c-e12e0c003dbf admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:04.760737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d28fd23e-5a0d-43fd-aa5c-e12e0c003dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:04.794125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28fd23e-5a0d-43fd-aa5c-e12e0c003dbf admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:04.904171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:04.904894 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:04.905142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:04.905475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] PATH_INFO: `/v3/projects/44210f20f39e406bb7203ce6f31bee58` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:04.908454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44210f20f39e406bb7203ce6f31bee58)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:04.910724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:04.937485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:05.019437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44210f20f39e406bb7203ce6f31bee58', 'request_id': 'req-3bdffef4-d1a3-4158-93df-976ab06226f7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:05.032578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44210f20f39e406bb7203ce6f31bee58', 'request_id': 'req-3bdffef4-d1a3-4158-93df-976ab06226f7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:05.047128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44210f20f39e406bb7203ce6f31bee58', 'request_id': 'req-3bdffef4-d1a3-4158-93df-976ab06226f7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:05.065939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] The token cache is being invalidate because project 44210f20f39e406bb7203ce6f31bee58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:30:05.066229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:05.067079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3bdffef4-d1a3-4158-93df-976ab06226f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:30:05.072091 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1110/2203] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:30:04 2022] DELETE /identity/v3/projects/44210f20f39e406bb7203ce6f31bee58 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:30:05.078856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6aef9-d66d-4839-8b47-94743dc42c6a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:05.079318 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6aef9-d66d-4839-8b47-94743dc42c6a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:05.079434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6aef9-d66d-4839-8b47-94743dc42c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:05.125160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0cc6aef9-d66d-4839-8b47-94743dc42c6a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:05.128294 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0cc6aef9-d66d-4839-8b47-94743dc42c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:05.172838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:05.270348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1094/2204] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:30:05.282278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa02ce-d778-47b9-aa53-5bcfaab592b9 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:05.282645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa02ce-d778-47b9-aa53-5bcfaab592b9 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:05.282879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa02ce-d778-47b9-aa53-5bcfaab592b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:05.312287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3daa02ce-d778-47b9-aa53-5bcfaab592b9 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:05.314556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3daa02ce-d778-47b9-aa53-5bcfaab592b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:05.324964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1111/2205] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:30:05.330268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6aef9-d66d-4839-8b47-94743dc42c6a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:05.341378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6aef9-d66d-4839-8b47-94743dc42c6a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:05.348292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec111903-d944-4158-a393-d6ef0b90ef83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:05.349073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111903-d944-4158-a393-d6ef0b90ef83 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:05.349531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111903-d944-4158-a393-d6ef0b90ef83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:05.349817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec111903-d944-4158-a393-d6ef0b90ef83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:05.352706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec111903-d944-4158-a393-d6ef0b90ef83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:30:05.352706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec111903-d944-4158-a393-d6ef0b90ef83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:05.352706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec111903-d944-4158-a393-d6ef0b90ef83 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:05.370130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1095/2206] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:30:05.374909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa02ce-d778-47b9-aa53-5bcfaab592b9 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:05.390309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa02ce-d778-47b9-aa53-5bcfaab592b9 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:05.395428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c67e9-f11d-4948-86c6-9439aacb9e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:05.397073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c67e9-f11d-4948-86c6-9439aacb9e23 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:05.397664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c67e9-f11d-4948-86c6-9439aacb9e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:05.398165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492c67e9-f11d-4948-86c6-9439aacb9e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:05.399390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c67e9-f11d-4948-86c6-9439aacb9e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:05.402430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492c67e9-f11d-4948-86c6-9439aacb9e23 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:05.440422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1112/2207] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:30:05 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:30:05.446318 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec111903-d944-4158-a393-d6ef0b90ef83 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:05.473470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec111903-d944-4158-a393-d6ef0b90ef83 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:05.570939 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500346e-d28b-44b9-9ae8-de7216687f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:05.571646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500346e-d28b-44b9-9ae8-de7216687f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:05.572223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500346e-d28b-44b9-9ae8-de7216687f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:05.572505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a500346e-d28b-44b9-9ae8-de7216687f99 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:05.573223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500346e-d28b-44b9-9ae8-de7216687f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:05.575479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a500346e-d28b-44b9-9ae8-de7216687f99 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:05.605991 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1096/2208] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:30:05 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:30:05.611575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c67e9-f11d-4948-86c6-9439aacb9e23 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:05.623959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492c67e9-f11d-4948-86c6-9439aacb9e23 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:05.629823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7537a7c-54f2-42e8-b700-b9e0a9e0b88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:05.630710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7537a7c-54f2-42e8-b700-b9e0a9e0b88b admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:05.631063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7537a7c-54f2-42e8-b700-b9e0a9e0b88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:05.631653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7537a7c-54f2-42e8-b700-b9e0a9e0b88b admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:05.632980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7537a7c-54f2-42e8-b700-b9e0a9e0b88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:05.635333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7537a7c-54f2-42e8-b700-b9e0a9e0b88b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:05.649943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1113/2209] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:30:05.654211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500346e-d28b-44b9-9ae8-de7216687f99 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:05.668177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a500346e-d28b-44b9-9ae8-de7216687f99 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:05.675648 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ad83f-f240-4cbd-87df-ed99af421fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:05.676606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ad83f-f240-4cbd-87df-ed99af421fae admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:05.677087 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ad83f-f240-4cbd-87df-ed99af421fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:05.677513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ad83f-f240-4cbd-87df-ed99af421fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:05.678817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ad83f-f240-4cbd-87df-ed99af421fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:05.681525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ad83f-f240-4cbd-87df-ed99af421fae admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:05.704506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7537a7c-54f2-42e8-b700-b9e0a9e0b88b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:05.705672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1097/2210] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:30:05.717988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7537a7c-54f2-42e8-b700-b9e0a9e0b88b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:05.722938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6457139-11c7-4f66-8095-e3c29e72a7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:05.723910 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6457139-11c7-4f66-8095-e3c29e72a7d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:05.724360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6457139-11c7-4f66-8095-e3c29e72a7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:05.724641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6457139-11c7-4f66-8095-e3c29e72a7d3 admin admin] PATH_INFO: `/v3/projects/5732783c305f47a389104f7ea5ac5fcf/users/482e2d3af7994b6692af6a4fa17dde1d/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:05.756497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6457139-11c7-4f66-8095-e3c29e72a7d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5732783c305f47a389104f7ea5ac5fcf, user_id=482e2d3af7994b6692af6a4fa17dde1d, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:05.761559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6457139-11c7-4f66-8095-e3c29e72a7d3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:05.787875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189715a-4c08-4ab9-9b38-af452e2372c3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:05.788727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189715a-4c08-4ab9-9b38-af452e2372c3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:05.789393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189715a-4c08-4ab9-9b38-af452e2372c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:05.791972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1114/2211] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:30:05 2022] PUT /identity/v3/projects/5732783c305f47a389104f7ea5ac5fcf/users/482e2d3af7994b6692af6a4fa17dde1d/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:30:05.816062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7189715a-4c08-4ab9-9b38-af452e2372c3 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:05.817530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7189715a-4c08-4ab9-9b38-af452e2372c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:06.013171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1098/2212] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:30:06.033506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6457139-11c7-4f66-8095-e3c29e72a7d3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:06.036912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f6457139-11c7-4f66-8095-e3c29e72a7d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:06.171079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6457139-11c7-4f66-8095-e3c29e72a7d3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:06.179838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd166a-2612-42eb-9ab1-ed83006b29fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:06.181222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd166a-2612-42eb-9ab1-ed83006b29fa service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:06.181729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd166a-2612-42eb-9ab1-ed83006b29fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:06.182238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd166a-2612-42eb-9ab1-ed83006b29fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:06.207083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd166a-2612-42eb-9ab1-ed83006b29fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:06.210170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bd166a-2612-42eb-9ab1-ed83006b29fa service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:06.231421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1115/2213] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:30:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:30:14.205224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2c095-c4f8-4be4-9faf-3023f2a42470 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:14.206573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2c095-c4f8-4be4-9faf-3023f2a42470 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:14.207688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a2c095-c4f8-4be4-9faf-3023f2a42470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:14.310010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-54a2c095-c4f8-4be4-9faf-3023f2a42470 None None] MFA Rules not processed for user `50d228e676244ec8b45b978665a7b2da`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:14.312484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-54a2c095-c4f8-4be4-9faf-3023f2a42470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:14.352956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:14.436863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1099/2214] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:30:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:30:14.455471 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd166a-2612-42eb-9ab1-ed83006b29fa service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:14.464940 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-56bd166a-2612-42eb-9ab1-ed83006b29fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:14.619711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:14.702877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd166a-2612-42eb-9ab1-ed83006b29fa service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:14.734122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be57605-35c1-4061-969e-04f4dd4ae0f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:14.736902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57605-35c1-4061-969e-04f4dd4ae0f3 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:14.737754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57605-35c1-4061-969e-04f4dd4ae0f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:14.739249 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be57605-35c1-4061-969e-04f4dd4ae0f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:14.743945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4be57605-35c1-4061-969e-04f4dd4ae0f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:14.882446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be57605-35c1-4061-969e-04f4dd4ae0f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:14.884862 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be57605-35c1-4061-969e-04f4dd4ae0f3 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:14.901734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1116/2215] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:30:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:30:15.392328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bd9dc-2ae3-4027-a5dc-bea45ed2dcfc None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:15.392806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bd9dc-2ae3-4027-a5dc-bea45ed2dcfc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:15.392992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bd9dc-2ae3-4027-a5dc-bea45ed2dcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:15.442695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-025bd9dc-2ae3-4027-a5dc-bea45ed2dcfc None None] MFA Rules not processed for user `482e2d3af7994b6692af6a4fa17dde1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:15.444573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-025bd9dc-2ae3-4027-a5dc-bea45ed2dcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:15.577388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1100/2216] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:30:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:30:15.582061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be57605-35c1-4061-969e-04f4dd4ae0f3 service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:15.607844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be57605-35c1-4061-969e-04f4dd4ae0f3 service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:15.614863 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6dcb3-d7a9-4ae3-ab69-db81a2959ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:15.616089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6dcb3-d7a9-4ae3-ab69-db81a2959ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:15.616643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6dcb3-d7a9-4ae3-ab69-db81a2959ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:15.617428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6dcb3-d7a9-4ae3-ab69-db81a2959ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:15.618516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6dcb3-d7a9-4ae3-ab69-db81a2959ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:15.621351 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6dcb3-d7a9-4ae3-ab69-db81a2959ae2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:15.667185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1117/2217] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:30:15 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:30:15.672417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bd9dc-2ae3-4027-a5dc-bea45ed2dcfc None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:15.690374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025bd9dc-2ae3-4027-a5dc-bea45ed2dcfc None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:15.834674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51010791-4ba1-4a45-a60e-535be42763f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:15.836661 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51010791-4ba1-4a45-a60e-535be42763f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:15.837626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51010791-4ba1-4a45-a60e-535be42763f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:15.838353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51010791-4ba1-4a45-a60e-535be42763f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:15.840736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51010791-4ba1-4a45-a60e-535be42763f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:15.846675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51010791-4ba1-4a45-a60e-535be42763f3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:15.912156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1101/2218] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:30:15 2022] POST /identity/v3/users => generated 299 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:30:15.918842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6dcb3-d7a9-4ae3-ab69-db81a2959ae2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:15.940108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6dcb3-d7a9-4ae3-ab69-db81a2959ae2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:15.950958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594ed42-96f1-4674-a5e2-6c54d7f7b903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:15.952563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594ed42-96f1-4674-a5e2-6c54d7f7b903 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:15.953347 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594ed42-96f1-4674-a5e2-6c54d7f7b903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:15.953962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594ed42-96f1-4674-a5e2-6c54d7f7b903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:15.955789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594ed42-96f1-4674-a5e2-6c54d7f7b903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:15.959533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594ed42-96f1-4674-a5e2-6c54d7f7b903 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:15.978645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1118/2219] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:30:15.981941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51010791-4ba1-4a45-a60e-535be42763f3 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:15.995692 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51010791-4ba1-4a45-a60e-535be42763f3 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:16.010462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8b6e-48a4-4061-bb30-63f86bd783a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:16.014515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8b6e-48a4-4061-bb30-63f86bd783a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:16.014903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8b6e-48a4-4061-bb30-63f86bd783a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:16.015195 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8b6e-48a4-4061-bb30-63f86bd783a7 admin admin] PATH_INFO: `/v3/projects/d6bb6c3f331a4fb581256890765018ac/users/a8f23076a49249048ca9dfc73547c8f2/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:16.044948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8b6e-48a4-4061-bb30-63f86bd783a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6bb6c3f331a4fb581256890765018ac, user_id=a8f23076a49249048ca9dfc73547c8f2, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:16.048131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8b6e-48a4-4061-bb30-63f86bd783a7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:16.066291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1102/2220] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:30:15 2022] PUT /identity/v3/projects/d6bb6c3f331a4fb581256890765018ac/users/a8f23076a49249048ca9dfc73547c8f2/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:30:16.071905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594ed42-96f1-4674-a5e2-6c54d7f7b903 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:16.090471 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594ed42-96f1-4674-a5e2-6c54d7f7b903 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:16.184055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb9bef6-9360-4386-8ec2-7e144e35f433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:16.185587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9bef6-9360-4386-8ec2-7e144e35f433 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:16.185947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9bef6-9360-4386-8ec2-7e144e35f433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:16.187032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb9bef6-9360-4386-8ec2-7e144e35f433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:16.187986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9bef6-9360-4386-8ec2-7e144e35f433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:16.190210 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb9bef6-9360-4386-8ec2-7e144e35f433 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:16.203079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1119/2221] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:30:16.208062 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8b6e-48a4-4061-bb30-63f86bd783a7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:16.237006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8b6e-48a4-4061-bb30-63f86bd783a7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:16.243794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76352387-9197-4bba-9f5e-7e2b296eccb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:16.245495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76352387-9197-4bba-9f5e-7e2b296eccb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:16.245495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76352387-9197-4bba-9f5e-7e2b296eccb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:16.245810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76352387-9197-4bba-9f5e-7e2b296eccb8 admin admin] PATH_INFO: `/v3/projects/d6bb6c3f331a4fb581256890765018ac/users/a8f23076a49249048ca9dfc73547c8f2/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:16.248020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76352387-9197-4bba-9f5e-7e2b296eccb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6bb6c3f331a4fb581256890765018ac, user_id=a8f23076a49249048ca9dfc73547c8f2, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:16.250766 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76352387-9197-4bba-9f5e-7e2b296eccb8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:16.258404 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:16.271925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1103/2222] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:30:16 2022] PUT /identity/v3/projects/d6bb6c3f331a4fb581256890765018ac/users/a8f23076a49249048ca9dfc73547c8f2/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:30:17.170992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca31043-16a6-46d0-a3db-b2a9c22887d8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:17.171554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca31043-16a6-46d0-a3db-b2a9c22887d8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:17.171554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca31043-16a6-46d0-a3db-b2a9c22887d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:17.207743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8ca31043-16a6-46d0-a3db-b2a9c22887d8 None None] MFA Rules not processed for user `9c3ca706dfed4770a1eef107b28a9233`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:17.208931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8ca31043-16a6-46d0-a3db-b2a9c22887d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:17.304092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:17.347931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1120/2223] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:30:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:30:21.336097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6431db-62bf-453d-ba5c-1347f3cb86e7 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:21.336741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6431db-62bf-453d-ba5c-1347f3cb86e7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:21.336895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6431db-62bf-453d-ba5c-1347f3cb86e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:21.367745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2f6431db-62bf-453d-ba5c-1347f3cb86e7 None None] MFA Rules not processed for user `19024dea6ca94745b38158beb08f29fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:21.370088 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2f6431db-62bf-453d-ba5c-1347f3cb86e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:21.530442 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1104/2224] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:30:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:30:22.139630 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642b686-6064-45f3-b294-13ab4c29a20d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:22.140087 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642b686-6064-45f3-b294-13ab4c29a20d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:22.140182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642b686-6064-45f3-b294-13ab4c29a20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:22.206839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f642b686-6064-45f3-b294-13ab4c29a20d None None] MFA Rules not processed for user `a8f23076a49249048ca9dfc73547c8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:22.215775 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f642b686-6064-45f3-b294-13ab4c29a20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:22.402717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1121/2225] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:30:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:30:22.418283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6431db-62bf-453d-ba5c-1347f3cb86e7 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:22.424064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2f6431db-62bf-453d-ba5c-1347f3cb86e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:22.545758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6431db-62bf-453d-ba5c-1347f3cb86e7 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:22.554444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3b087-f932-4d92-9bc2-13a879a7526a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3778997232d44cd3a3c4fb3d53e4db53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:22.556135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3b087-f932-4d92-9bc2-13a879a7526a service glance] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:22.556682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3b087-f932-4d92-9bc2-13a879a7526a service glance] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:22.557404 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3b087-f932-4d92-9bc2-13a879a7526a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:22.682467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3b087-f932-4d92-9bc2-13a879a7526a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:22.685205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3b087-f932-4d92-9bc2-13a879a7526a service glance] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:22.696984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1105/2226] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:30:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:30:22.863060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1c3a-5f25-4ef0-a607-cfd8138dd337 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:22.863453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1c3a-5f25-4ef0-a607-cfd8138dd337 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:22.863720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb1c3a-5f25-4ef0-a607-cfd8138dd337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:22.889655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bbfb1c3a-5f25-4ef0-a607-cfd8138dd337 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:22.892243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bbfb1c3a-5f25-4ef0-a607-cfd8138dd337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:22.991973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1122/2227] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:30:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:30:23.032620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3b087-f932-4d92-9bc2-13a879a7526a service glance] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:23.033869 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e5d3b087-f932-4d92-9bc2-13a879a7526a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:23.128603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:23.166129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3b087-f932-4d92-9bc2-13a879a7526a service glance] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:23.181689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc9e4f-baaa-4de7-a601-634941a5e5d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:23.183612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc9e4f-baaa-4de7-a601-634941a5e5d6 service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:23.183918 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc9e4f-baaa-4de7-a601-634941a5e5d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:23.184202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc9e4f-baaa-4de7-a601-634941a5e5d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:23.195770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc9e4f-baaa-4de7-a601-634941a5e5d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:23.197334 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc9e4f-baaa-4de7-a601-634941a5e5d6 service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:23.209717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1106/2228] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:30:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 08:30:32.984334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=66 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:33.204794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:36.331021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a59cb-8c78-49e4-82d2-15b5b55ad39e None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:36.331786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a59cb-8c78-49e4-82d2-15b5b55ad39e None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:36.331786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a59cb-8c78-49e4-82d2-15b5b55ad39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:36.358674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-764a59cb-8c78-49e4-82d2-15b5b55ad39e None None] MFA Rules not processed for user `482e2d3af7994b6692af6a4fa17dde1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:36.360405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-764a59cb-8c78-49e4-82d2-15b5b55ad39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:36.371139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1123/2229] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:30:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:30:39.003348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc9e4f-baaa-4de7-a601-634941a5e5d6 service swift] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:39.005755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a7bc9e4f-baaa-4de7-a601-634941a5e5d6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:39.116541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc9e4f-baaa-4de7-a601-634941a5e5d6 service swift] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:39.122743 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b63fe-5b91-4345-be42-b1bbbc2a4f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:39.123849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b63fe-5b91-4345-be42-b1bbbc2a4f34 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:39.124272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b63fe-5b91-4345-be42-b1bbbc2a4f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:39.124738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b63fe-5b91-4345-be42-b1bbbc2a4f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:39.127646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7e1b63fe-5b91-4345-be42-b1bbbc2a4f34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:39.227916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b63fe-5b91-4345-be42-b1bbbc2a4f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:39.229542 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b63fe-5b91-4345-be42-b1bbbc2a4f34 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:39.243640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1107/2230] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:30:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:30:40.363674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bf387-f10a-43fc-92d0-6e7db53e610b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:40.364524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bf387-f10a-43fc-92d0-6e7db53e610b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:40.364986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bf387-f10a-43fc-92d0-6e7db53e610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:40.399575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-250bf387-f10a-43fc-92d0-6e7db53e610b None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:30:40.402981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-250bf387-f10a-43fc-92d0-6e7db53e610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:40.418494 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b63fe-5b91-4345-be42-b1bbbc2a4f34 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:40.419320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1124/2231] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:30:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:30:40.434244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b63fe-5b91-4345-be42-b1bbbc2a4f34 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:40.435893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bf387-f10a-43fc-92d0-6e7db53e610b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:40.439488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121c863-4ac8-4b4c-87ef-3065d19d187a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:40.440440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121c863-4ac8-4b4c-87ef-3065d19d187a service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:40.440781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121c863-4ac8-4b4c-87ef-3065d19d187a service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:40.441059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4121c863-4ac8-4b4c-87ef-3065d19d187a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:40.443237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4121c863-4ac8-4b4c-87ef-3065d19d187a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:40.464920 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bf387-f10a-43fc-92d0-6e7db53e610b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:40.474438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55543862-0061-4482-ba07-5b21b653ab6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:40.476251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55543862-0061-4482-ba07-5b21b653ab6a service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:40.476545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55543862-0061-4482-ba07-5b21b653ab6a service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:40.476821 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55543862-0061-4482-ba07-5b21b653ab6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:40.484708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121c863-4ac8-4b4c-87ef-3065d19d187a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:40.486513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4121c863-4ac8-4b4c-87ef-3065d19d187a service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:40.493485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55543862-0061-4482-ba07-5b21b653ab6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:40.496115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55543862-0061-4482-ba07-5b21b653ab6a service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:40.503185 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1108/2232] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:30:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:30:40.523521 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1125/2233] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:30:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:30:49.565215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121c863-4ac8-4b4c-87ef-3065d19d187a service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:30:49.566900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4121c863-4ac8-4b4c-87ef-3065d19d187a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:49.584021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4121c863-4ac8-4b4c-87ef-3065d19d187a service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:30:49.591458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db09e4-f3b4-4b14-ad2d-9f0f8962b91e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:30:49.592147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db09e4-f3b4-4b14-ad2d-9f0f8962b91e service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:30:49.592439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db09e4-f3b4-4b14-ad2d-9f0f8962b91e service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:30:49.592717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db09e4-f3b4-4b14-ad2d-9f0f8962b91e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:30:49.594551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c3db09e4-f3b4-4b14-ad2d-9f0f8962b91e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:30:49.663902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db09e4-f3b4-4b14-ad2d-9f0f8962b91e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:30:49.666176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db09e4-f3b4-4b14-ad2d-9f0f8962b91e service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:30:49.677739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1109/2234] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:30:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:30:50.502112 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=12 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:30:59.672741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=88 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:31:02.460037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55543862-0061-4482-ba07-5b21b653ab6a service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:02.470791 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55543862-0061-4482-ba07-5b21b653ab6a service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:02.474822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71f080-6629-4135-8d6f-1515dbbe7bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:02.475494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71f080-6629-4135-8d6f-1515dbbe7bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:02.475935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71f080-6629-4135-8d6f-1515dbbe7bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:02.476435 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71f080-6629-4135-8d6f-1515dbbe7bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:02.478928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cf71f080-6629-4135-8d6f-1515dbbe7bc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:02.489325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71f080-6629-4135-8d6f-1515dbbe7bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:02.490906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf71f080-6629-4135-8d6f-1515dbbe7bc7 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:02.499505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1126/2235] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:05.505058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db09e4-f3b4-4b14-ad2d-9f0f8962b91e service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:05.516072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db09e4-f3b4-4b14-ad2d-9f0f8962b91e service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:05.521588 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129e4a6-cdf7-4bf1-aade-008ed4423ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:05.522666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129e4a6-cdf7-4bf1-aade-008ed4423ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:05.523116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129e4a6-cdf7-4bf1-aade-008ed4423ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:05.523544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3129e4a6-cdf7-4bf1-aade-008ed4423ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:05.525625 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3129e4a6-cdf7-4bf1-aade-008ed4423ea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:05.616500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129e4a6-cdf7-4bf1-aade-008ed4423ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:05.618894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3129e4a6-cdf7-4bf1-aade-008ed4423ea9 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:05.638512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1110/2236] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:11.847720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71f080-6629-4135-8d6f-1515dbbe7bc7 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:11.862892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71f080-6629-4135-8d6f-1515dbbe7bc7 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:11.869701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60062171-0f48-4a19-ad43-290c6e33d59a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:11.870368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60062171-0f48-4a19-ad43-290c6e33d59a service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:11.870697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60062171-0f48-4a19-ad43-290c6e33d59a service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:11.870978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60062171-0f48-4a19-ad43-290c6e33d59a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:11.874523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-60062171-0f48-4a19-ad43-290c6e33d59a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:11.888996 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60062171-0f48-4a19-ad43-290c6e33d59a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:11.890660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60062171-0f48-4a19-ad43-290c6e33d59a service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:11.907264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1127/2237] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:11.931717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129e4a6-cdf7-4bf1-aade-008ed4423ea9 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:11.933321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3129e4a6-cdf7-4bf1-aade-008ed4423ea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:11.997864 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3129e4a6-cdf7-4bf1-aade-008ed4423ea9 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:12.001474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fda96a-9c4b-4afd-a65c-b6eb47ab5743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:12.002121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fda96a-9c4b-4afd-a65c-b6eb47ab5743 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:12.002500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fda96a-9c4b-4afd-a65c-b6eb47ab5743 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:12.002714 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fda96a-9c4b-4afd-a65c-b6eb47ab5743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:12.004518 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-87fda96a-9c4b-4afd-a65c-b6eb47ab5743 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:12.065564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fda96a-9c4b-4afd-a65c-b6eb47ab5743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:12.067075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fda96a-9c4b-4afd-a65c-b6eb47ab5743 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:12.077999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1111/2238] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:16.178971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef8f88-72af-4811-97f3-a9570275b4b1 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:16.179487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef8f88-72af-4811-97f3-a9570275b4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:16.179610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef8f88-72af-4811-97f3-a9570275b4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:16.207520 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-c1ef8f88-72af-4811-97f3-a9570275b4b1 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:31:16.208760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c1ef8f88-72af-4811-97f3-a9570275b4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:16.217322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1128/2239] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:31:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:31:16.232269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fda96a-9c4b-4afd-a65c-b6eb47ab5743 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:16.246521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fda96a-9c4b-4afd-a65c-b6eb47ab5743 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:16.251849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace624b-1865-499e-8c50-e4c2a11e7023 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:16.252736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace624b-1865-499e-8c50-e4c2a11e7023 service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:16.253123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace624b-1865-499e-8c50-e4c2a11e7023 service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:16.253522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace624b-1865-499e-8c50-e4c2a11e7023 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:16.265890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace624b-1865-499e-8c50-e4c2a11e7023 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:16.272907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace624b-1865-499e-8c50-e4c2a11e7023 service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:16.285608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1112/2240] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:31:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:31:18.226951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef8f88-72af-4811-97f3-a9570275b4b1 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:18.242167 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef8f88-72af-4811-97f3-a9570275b4b1 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:18.249608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740376c-325c-45be-ae81-bd09b59e4f18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:18.250272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740376c-325c-45be-ae81-bd09b59e4f18 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:18.250561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740376c-325c-45be-ae81-bd09b59e4f18 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:18.250883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740376c-325c-45be-ae81-bd09b59e4f18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:18.252711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b740376c-325c-45be-ae81-bd09b59e4f18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:18.316092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740376c-325c-45be-ae81-bd09b59e4f18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:18.317751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740376c-325c-45be-ae81-bd09b59e4f18 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:18.327282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1129/2241] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:22.879036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e42ab-1d9b-4bff-835e-856c1344c776 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:22.879645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e42ab-1d9b-4bff-835e-856c1344c776 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:22.879645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e42ab-1d9b-4bff-835e-856c1344c776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:22.897952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-599e42ab-1d9b-4bff-835e-856c1344c776 None None] MFA Rules not processed for user `e6c2070c6d2a450c887b8ca6b97c4b38`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:31:22.899201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-599e42ab-1d9b-4bff-835e-856c1344c776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:22.906828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1113/2242] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:31:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:31:23.244707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740376c-325c-45be-ae81-bd09b59e4f18 service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:23.258007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740376c-325c-45be-ae81-bd09b59e4f18 service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:23.266475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd24db1-b641-4dd2-850b-64a2a07fb3f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:23.267184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd24db1-b641-4dd2-850b-64a2a07fb3f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:23.267481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd24db1-b641-4dd2-850b-64a2a07fb3f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:23.267768 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd24db1-b641-4dd2-850b-64a2a07fb3f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:23.277596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4fd24db1-b641-4dd2-850b-64a2a07fb3f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:23.291648 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd24db1-b641-4dd2-850b-64a2a07fb3f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:23.293329 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd24db1-b641-4dd2-850b-64a2a07fb3f4 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:23.309071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1130/2243] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:23.669959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e42ab-1d9b-4bff-835e-856c1344c776 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:23.672138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-599e42ab-1d9b-4bff-835e-856c1344c776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:23.721526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:31:23.784679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e42ab-1d9b-4bff-835e-856c1344c776 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:23.789915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40360977-acee-43b2-bff1-8ba1f7be41dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:23.790963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40360977-acee-43b2-bff1-8ba1f7be41dd service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:23.791369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40360977-acee-43b2-bff1-8ba1f7be41dd service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:23.791819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40360977-acee-43b2-bff1-8ba1f7be41dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:23.793904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-40360977-acee-43b2-bff1-8ba1f7be41dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:23.815783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40360977-acee-43b2-bff1-8ba1f7be41dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:23.818625 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40360977-acee-43b2-bff1-8ba1f7be41dd service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:23.836064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1114/2244] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:33.298848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=49 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:31:33.826456 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=19 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:31:37.654222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd24db1-b641-4dd2-850b-64a2a07fb3f4 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:37.663109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd24db1-b641-4dd2-850b-64a2a07fb3f4 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:37.667797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02c5e9-d508-4df5-84cf-e103d64d9c06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:37.668664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02c5e9-d508-4df5-84cf-e103d64d9c06 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:37.669129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02c5e9-d508-4df5-84cf-e103d64d9c06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:37.669567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02c5e9-d508-4df5-84cf-e103d64d9c06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:37.682608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02c5e9-d508-4df5-84cf-e103d64d9c06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:37.684125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02c5e9-d508-4df5-84cf-e103d64d9c06 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:37.699027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1131/2245] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:39.572188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40360977-acee-43b2-bff1-8ba1f7be41dd service placement] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:39.605467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40360977-acee-43b2-bff1-8ba1f7be41dd service placement] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:39.613275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbb9cb-97f8-4e8f-ab64-7f11ae1f7d0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:39.614083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbb9cb-97f8-4e8f-ab64-7f11ae1f7d0a service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:39.614623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbb9cb-97f8-4e8f-ab64-7f11ae1f7d0a service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:39.614957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbb9cb-97f8-4e8f-ab64-7f11ae1f7d0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:39.617045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6cbbb9cb-97f8-4e8f-ab64-7f11ae1f7d0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:39.631528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbb9cb-97f8-4e8f-ab64-7f11ae1f7d0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:39.633511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbb9cb-97f8-4e8f-ab64-7f11ae1f7d0a service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:39.646355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1115/2246] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:40.920330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02c5e9-d508-4df5-84cf-e103d64d9c06 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:40.941221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02c5e9-d508-4df5-84cf-e103d64d9c06 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:40.951603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53d9b9-6d66-43e9-bf91-7dcfbc8acbb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:40.952451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53d9b9-6d66-43e9-bf91-7dcfbc8acbb0 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:40.952857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53d9b9-6d66-43e9-bf91-7dcfbc8acbb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:40.953210 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b53d9b9-6d66-43e9-bf91-7dcfbc8acbb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:40.955706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6b53d9b9-6d66-43e9-bf91-7dcfbc8acbb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:40.970144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53d9b9-6d66-43e9-bf91-7dcfbc8acbb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:40.972113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b53d9b9-6d66-43e9-bf91-7dcfbc8acbb0 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:41.003184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1132/2247] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:41.683064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbb9cb-97f8-4e8f-ab64-7f11ae1f7d0a service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:41.709344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbb9cb-97f8-4e8f-ab64-7f11ae1f7d0a service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:41.716545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3abeb-02b5-4632-92a3-161574e886e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:41.717341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3abeb-02b5-4632-92a3-161574e886e4 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:41.717639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3abeb-02b5-4632-92a3-161574e886e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:41.717926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3abeb-02b5-4632-92a3-161574e886e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:41.722074 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0bf3abeb-02b5-4632-92a3-161574e886e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:41.800347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3abeb-02b5-4632-92a3-161574e886e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:41.802324 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3abeb-02b5-4632-92a3-161574e886e4 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:41.814057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1116/2248] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:44.640736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53d9b9-6d66-43e9-bf91-7dcfbc8acbb0 service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:44.650812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b53d9b9-6d66-43e9-bf91-7dcfbc8acbb0 service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:44.654872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13094049-05ee-48fa-8f95-d79101a3a6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:44.655526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13094049-05ee-48fa-8f95-d79101a3a6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:44.655892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13094049-05ee-48fa-8f95-d79101a3a6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:44.656179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13094049-05ee-48fa-8f95-d79101a3a6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:44.657964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-13094049-05ee-48fa-8f95-d79101a3a6a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:44.669455 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13094049-05ee-48fa-8f95-d79101a3a6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:44.670571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13094049-05ee-48fa-8f95-d79101a3a6a8 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:44.681619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1133/2249] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:44.829789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07158333-33a7-49d8-96cf-99f8a421da3e None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:44.830340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07158333-33a7-49d8-96cf-99f8a421da3e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:44.830755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07158333-33a7-49d8-96cf-99f8a421da3e None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:44.832601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1117/2250] 10.222.0.64 () {54 vars in 921 bytes} [Wed Jul 20 08:31:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:31:44.843731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905cb07-4afc-4ac4-9c63-4f4fcadcacde None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:44.844449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905cb07-4afc-4ac4-9c63-4f4fcadcacde None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:44.845533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905cb07-4afc-4ac4-9c63-4f4fcadcacde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:44.883595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a905cb07-4afc-4ac4-9c63-4f4fcadcacde None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:31:44.886571 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a905cb07-4afc-4ac4-9c63-4f4fcadcacde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:44.897525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1134/2251] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Jul 20 08:31:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:31:44.912503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07158333-33a7-49d8-96cf-99f8a421da3e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:44.914332 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-07158333-33a7-49d8-96cf-99f8a421da3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:44.929955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07158333-33a7-49d8-96cf-99f8a421da3e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:44.935189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec8740e-0e76-44e5-acd6-d2b1ff4a97ff service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3778997232d44cd3a3c4fb3d53e4db53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:44.936024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8740e-0e76-44e5-acd6-d2b1ff4a97ff service glance] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:44.936538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8740e-0e76-44e5-acd6-d2b1ff4a97ff service glance] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:44.936999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec8740e-0e76-44e5-acd6-d2b1ff4a97ff service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:44.951316 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8740e-0e76-44e5-acd6-d2b1ff4a97ff service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:44.952758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec8740e-0e76-44e5-acd6-d2b1ff4a97ff service glance] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:44.970301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1118/2252] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:45.044307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba59483-302d-47c4-bdd6-72b66cc36f22 None None] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:45.044692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba59483-302d-47c4-bdd6-72b66cc36f22 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:45.044979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba59483-302d-47c4-bdd6-72b66cc36f22 None None] PATH_INFO: `/` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:45.046366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1135/2253] 10.222.0.64 () {54 vars in 921 bytes} [Wed Jul 20 08:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:31:45.059445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8030d5-e8ba-46e7-95f9-042d85d4eaf4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:45.059973 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8030d5-e8ba-46e7-95f9-042d85d4eaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:45.061662 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8030d5-e8ba-46e7-95f9-042d85d4eaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:45.098433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3b8030d5-e8ba-46e7-95f9-042d85d4eaf4 None None] MFA Rules not processed for user `a3b03383554346138b98becb2828851a`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:31:45.099912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3b8030d5-e8ba-46e7-95f9-042d85d4eaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:45.107913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1119/2254] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Jul 20 08:31:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:31:45.121140 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba59483-302d-47c4-bdd6-72b66cc36f22 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:45.132347 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba59483-302d-47c4-bdd6-72b66cc36f22 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:45.136243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ef3d9-1707-4e9b-964c-6de849c6cc7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:45.136886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ef3d9-1707-4e9b-964c-6de849c6cc7b service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:45.137529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ef3d9-1707-4e9b-964c-6de849c6cc7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:45.137822 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ef3d9-1707-4e9b-964c-6de849c6cc7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:45.146151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ef3d9-1707-4e9b-964c-6de849c6cc7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:45.147688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ef3d9-1707-4e9b-964c-6de849c6cc7b service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:45.157138 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1136/2255] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:45.285030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677f034-02b1-4c57-b422-fbdd2c468e81 None None] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:45.285353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677f034-02b1-4c57-b422-fbdd2c468e81 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:45.285599 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5677f034-02b1-4c57-b422-fbdd2c468e81 None None] PATH_INFO: `/` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:45.287152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1120/2256] 10.222.0.64 () {54 vars in 921 bytes} [Wed Jul 20 08:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 08:31:45.304268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12204eb2-3573-4bd7-be09-c5ef5f50cb4c None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:45.304268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12204eb2-3573-4bd7-be09-c5ef5f50cb4c None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:45.304268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12204eb2-3573-4bd7-be09-c5ef5f50cb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:45.316846 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-12204eb2-3573-4bd7-be09-c5ef5f50cb4c None None] MFA Rules not processed for user `84f0da8709624834976d0662aeb02ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:31:45.318375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-12204eb2-3573-4bd7-be09-c5ef5f50cb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:45.328744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1137/2257] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Jul 20 08:31:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:31:45.341051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677f034-02b1-4c57-b422-fbdd2c468e81 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:45.352101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5677f034-02b1-4c57-b422-fbdd2c468e81 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:45.356992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558df3c-213e-40f4-8fe9-e83eb2a6927c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:45.357637 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558df3c-213e-40f4-8fe9-e83eb2a6927c service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:45.357976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558df3c-213e-40f4-8fe9-e83eb2a6927c service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:45.358255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558df3c-213e-40f4-8fe9-e83eb2a6927c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:45.370051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558df3c-213e-40f4-8fe9-e83eb2a6927c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:45.371902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558df3c-213e-40f4-8fe9-e83eb2a6927c service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:45.383305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1121/2258] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:49.242168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12204eb2-3573-4bd7-be09-c5ef5f50cb4c None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:49.252579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12204eb2-3573-4bd7-be09-c5ef5f50cb4c None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:49.255986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf4617-34b9-40ae-805f-e2cd2c2a20df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:49.256637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf4617-34b9-40ae-805f-e2cd2c2a20df service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:49.256924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf4617-34b9-40ae-805f-e2cd2c2a20df service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:49.257204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf4617-34b9-40ae-805f-e2cd2c2a20df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:49.321276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf4617-34b9-40ae-805f-e2cd2c2a20df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:49.323365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf4617-34b9-40ae-805f-e2cd2c2a20df service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:49.333859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1138/2259] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:31:49 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:31:55.378385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=49 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:31:56.509849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ccdb2-028a-43bb-878e-61d8ecd2b115 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:56.510702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ccdb2-028a-43bb-878e-61d8ecd2b115 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:56.510702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ccdb2-028a-43bb-878e-61d8ecd2b115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:56.528975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-113ccdb2-028a-43bb-878e-61d8ecd2b115 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:31:56.530203 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-113ccdb2-028a-43bb-878e-61d8ecd2b115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:31:56.539460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1122/2260] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:31:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:31:56.552012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf4617-34b9-40ae-805f-e2cd2c2a20df service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:31:56.564706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf4617-34b9-40ae-805f-e2cd2c2a20df service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:31:56.569377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f562357-2b93-4aa5-b7a6-a045783d69e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:31:56.569906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f562357-2b93-4aa5-b7a6-a045783d69e4 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:31:56.570394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f562357-2b93-4aa5-b7a6-a045783d69e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:31:56.570547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f562357-2b93-4aa5-b7a6-a045783d69e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:31:56.579500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f562357-2b93-4aa5-b7a6-a045783d69e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:31:56.581135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f562357-2b93-4aa5-b7a6-a045783d69e4 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:31:56.589813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1139/2261] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:31:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:32:06.523372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:32:06.586157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=66 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:32:22.833030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d74ba3-a76a-41ac-8264-ea23ad7ccdab None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:32:22.833664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d74ba3-a76a-41ac-8264-ea23ad7ccdab None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:32:22.833664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d74ba3-a76a-41ac-8264-ea23ad7ccdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:32:22.854397 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-06d74ba3-a76a-41ac-8264-ea23ad7ccdab None None] MFA Rules not processed for user `bdc9e59d03334b72a8052e0f1112523f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:32:22.855868 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-06d74ba3-a76a-41ac-8264-ea23ad7ccdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:32:22.863143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1123/2262] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:32:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:32:32.849164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:32:35.734833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93263f-c3b9-41d6-a79d-7267bf5e8681 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:32:35.735377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93263f-c3b9-41d6-a79d-7267bf5e8681 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:32:35.735377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93263f-c3b9-41d6-a79d-7267bf5e8681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:32:35.753406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7d93263f-c3b9-41d6-a79d-7267bf5e8681 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:32:35.754650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7d93263f-c3b9-41d6-a79d-7267bf5e8681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:32:35.762482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1140/2263] 10.222.0.64 () {58 vars in 967 bytes} [Wed Jul 20 08:32:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:32:35.784352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d74ba3-a76a-41ac-8264-ea23ad7ccdab None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:32:35.797603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d74ba3-a76a-41ac-8264-ea23ad7ccdab None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:32:35.802445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ffd12-9ff4-4c8e-80c5-5d5a1e60111f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:32:35.803135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ffd12-9ff4-4c8e-80c5-5d5a1e60111f service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:32:35.803420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ffd12-9ff4-4c8e-80c5-5d5a1e60111f service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:32:35.803699 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ffd12-9ff4-4c8e-80c5-5d5a1e60111f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:32:35.812063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ffd12-9ff4-4c8e-80c5-5d5a1e60111f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:32:35.813658 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ffd12-9ff4-4c8e-80c5-5d5a1e60111f service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:32:35.821895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1124/2264] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:32:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:32:45.747992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:32:45.818645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:07.618691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93263f-c3b9-41d6-a79d-7267bf5e8681 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:07.770382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93263f-c3b9-41d6-a79d-7267bf5e8681 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:07.777320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96a0a97-a875-4ba6-bb36-2e5b4753094c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:07.779139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a0a97-a875-4ba6-bb36-2e5b4753094c service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:07.779988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a0a97-a875-4ba6-bb36-2e5b4753094c service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:07.780507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96a0a97-a875-4ba6-bb36-2e5b4753094c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:07.783406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f96a0a97-a875-4ba6-bb36-2e5b4753094c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:07.916736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a0a97-a875-4ba6-bb36-2e5b4753094c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:07.926258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96a0a97-a875-4ba6-bb36-2e5b4753094c service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:07.973035 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1141/2265] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:34:10.885873 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bdaa3-55fe-4af8-8597-6fe49f4c743e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:10.886925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bdaa3-55fe-4af8-8597-6fe49f4c743e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:10.887604 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947bdaa3-55fe-4af8-8597-6fe49f4c743e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:10.920803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-947bdaa3-55fe-4af8-8597-6fe49f4c743e None None] MFA Rules not processed for user `19024dea6ca94745b38158beb08f29fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:10.922443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-947bdaa3-55fe-4af8-8597-6fe49f4c743e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:10.945157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1125/2266] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:15.108411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ff756-c27f-43ac-a9f1-ba5681626e1b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:15.109001 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ff756-c27f-43ac-a9f1-ba5681626e1b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:15.109001 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ff756-c27f-43ac-a9f1-ba5681626e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:15.128276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-521ff756-c27f-43ac-a9f1-ba5681626e1b None None] MFA Rules not processed for user `fb5f0f7e59dd4c548a6ea72bb25f1ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:15.129555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-521ff756-c27f-43ac-a9f1-ba5681626e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:15.136783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1142/2267] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:20.909833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:23.157232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bdaa3-55fe-4af8-8597-6fe49f4c743e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:23.179088 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947bdaa3-55fe-4af8-8597-6fe49f4c743e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:23.182872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcf2ca-d15e-4c27-993a-1fe00b01818c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:23.183544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcf2ca-d15e-4c27-993a-1fe00b01818c service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:23.183877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcf2ca-d15e-4c27-993a-1fe00b01818c service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:23.184157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcf2ca-d15e-4c27-993a-1fe00b01818c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:23.186198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c6dcf2ca-d15e-4c27-993a-1fe00b01818c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:23.289177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dcf2ca-d15e-4c27-993a-1fe00b01818c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:23.290683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dcf2ca-d15e-4c27-993a-1fe00b01818c service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:23.327370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1126/2268] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:34:23.886447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b67f7-b496-4aed-80f8-c0433a21cb49 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:23.888707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b67f7-b496-4aed-80f8-c0433a21cb49 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:23.889222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b67f7-b496-4aed-80f8-c0433a21cb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:23.913438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-117b67f7-b496-4aed-80f8-c0433a21cb49 None None] MFA Rules not processed for user `e6c2070c6d2a450c887b8ca6b97c4b38`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:23.916152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-117b67f7-b496-4aed-80f8-c0433a21cb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:23.931449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1143/2269] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:27.389491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcf2ca-d15e-4c27-993a-1fe00b01818c service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:27.405113 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcf2ca-d15e-4c27-993a-1fe00b01818c service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:27.412853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011203dc-824e-41b8-a790-e7b1f210059a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:27.413637 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011203dc-824e-41b8-a790-e7b1f210059a service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:27.414636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011203dc-824e-41b8-a790-e7b1f210059a service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:27.414990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011203dc-824e-41b8-a790-e7b1f210059a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:27.416931 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-011203dc-824e-41b8-a790-e7b1f210059a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:27.439995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011203dc-824e-41b8-a790-e7b1f210059a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:27.442181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011203dc-824e-41b8-a790-e7b1f210059a service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:27.454654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1127/2270] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:34:27.459290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b67f7-b496-4aed-80f8-c0433a21cb49 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:27.460805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-117b67f7-b496-4aed-80f8-c0433a21cb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:27.481242 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b67f7-b496-4aed-80f8-c0433a21cb49 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:27.491004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e8e9b-63ad-4293-b493-b802f3a7e968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:27.492018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e8e9b-63ad-4293-b493-b802f3a7e968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:27.492425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e8e9b-63ad-4293-b493-b802f3a7e968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:27.492614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e8e9b-63ad-4293-b493-b802f3a7e968 admin admin] PATH_INFO: `/v3/users/bdc9e59d03334b72a8052e0f1112523f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:27.493959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e8e9b-63ad-4293-b493-b802f3a7e968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc9e59d03334b72a8052e0f1112523f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:27.496045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e8e9b-63ad-4293-b493-b802f3a7e968 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:27.550978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-166e8e9b-63ad-4293-b493-b802f3a7e968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc9e59d03334b72a8052e0f1112523f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:27.563997 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-166e8e9b-63ad-4293-b493-b802f3a7e968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc9e59d03334b72a8052e0f1112523f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:27.564709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-166e8e9b-63ad-4293-b493-b802f3a7e968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc9e59d03334b72a8052e0f1112523f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:27.571804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-166e8e9b-63ad-4293-b493-b802f3a7e968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc9e59d03334b72a8052e0f1112523f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:27.579223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-166e8e9b-63ad-4293-b493-b802f3a7e968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc9e59d03334b72a8052e0f1112523f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:27.589173 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1144/2271] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:34:27 2022] DELETE /identity/v3/users/bdc9e59d03334b72a8052e0f1112523f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:27.849326 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011203dc-824e-41b8-a790-e7b1f210059a service placement] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:27.861934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011203dc-824e-41b8-a790-e7b1f210059a service placement] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:27.922410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:27.923335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:27.923736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:27.924020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] PATH_INFO: `/v3/projects/b044b1d7d5474351bd94eb1c90882933` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:27.925236 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b044b1d7d5474351bd94eb1c90882933)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:27.927220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:27.978880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:27.983259 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b044b1d7d5474351bd94eb1c90882933', 'request_id': 'req-f5bbf12f-8bd4-4fb6-b240-458e6c533168'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:27.993043 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b044b1d7d5474351bd94eb1c90882933', 'request_id': 'req-f5bbf12f-8bd4-4fb6-b240-458e6c533168'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:28.001099 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b044b1d7d5474351bd94eb1c90882933', 'request_id': 'req-f5bbf12f-8bd4-4fb6-b240-458e6c533168'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:28.018620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] The token cache is being invalidate because project b044b1d7d5474351bd94eb1c90882933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:34:28.018951 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:28.019592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f5bbf12f-8bd4-4fb6-b240-458e6c533168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:28.021180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1128/2272] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:34:27 2022] DELETE /identity/v3/projects/b044b1d7d5474351bd94eb1c90882933 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:28.033078 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d37909-4d51-4ae1-8925-bd83cf5c417a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:28.034913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d37909-4d51-4ae1-8925-bd83cf5c417a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:28.034913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d37909-4d51-4ae1-8925-bd83cf5c417a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:28.114968 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-57d37909-4d51-4ae1-8925-bd83cf5c417a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:28.116359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-57d37909-4d51-4ae1-8925-bd83cf5c417a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:28.129119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:28.196626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1145/2273] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:34:28.205164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e419e-3cc0-45f3-a5ec-8e8b5782c38e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:28.205459 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e419e-3cc0-45f3-a5ec-8e8b5782c38e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:28.205750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e419e-3cc0-45f3-a5ec-8e8b5782c38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:28.224957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-931e419e-3cc0-45f3-a5ec-8e8b5782c38e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:28.226190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-931e419e-3cc0-45f3-a5ec-8e8b5782c38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:28.235622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1129/2274] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:28.239777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d37909-4d51-4ae1-8925-bd83cf5c417a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:28.250300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d37909-4d51-4ae1-8925-bd83cf5c417a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:28.259050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0d4f7-dc10-4411-9fe3-ef7aca74abf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:28.259728 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d4f7-dc10-4411-9fe3-ef7aca74abf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:28.260013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d4f7-dc10-4411-9fe3-ef7aca74abf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:28.260290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0d4f7-dc10-4411-9fe3-ef7aca74abf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:28.263074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0d4f7-dc10-4411-9fe3-ef7aca74abf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:34:28.263395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0d4f7-dc10-4411-9fe3-ef7aca74abf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:28.265324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0d4f7-dc10-4411-9fe3-ef7aca74abf9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:28.281050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1146/2275] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:34:28.285296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e419e-3cc0-45f3-a5ec-8e8b5782c38e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:28.296449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e419e-3cc0-45f3-a5ec-8e8b5782c38e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:28.304031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12f015-04a2-4381-a9e6-3ae80dccbde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:28.304741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12f015-04a2-4381-a9e6-3ae80dccbde6 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:28.305030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12f015-04a2-4381-a9e6-3ae80dccbde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:28.305335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12f015-04a2-4381-a9e6-3ae80dccbde6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:28.306049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12f015-04a2-4381-a9e6-3ae80dccbde6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:28.308014 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12f015-04a2-4381-a9e6-3ae80dccbde6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:28.342373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1130/2276] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:34:28 2022] POST /identity/v3/projects => generated 357 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:28.344275 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0d4f7-dc10-4411-9fe3-ef7aca74abf9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:28.354729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0d4f7-dc10-4411-9fe3-ef7aca74abf9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:28.428478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e98265-21a9-45fb-91fa-417919a3a389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:28.429069 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e98265-21a9-45fb-91fa-417919a3a389 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:28.429357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e98265-21a9-45fb-91fa-417919a3a389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:28.429639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e98265-21a9-45fb-91fa-417919a3a389 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:28.430349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e98265-21a9-45fb-91fa-417919a3a389 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:28.432353 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e98265-21a9-45fb-91fa-417919a3a389 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:28.456331 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1147/2277] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:34:28 2022] POST /identity/v3/users => generated 279 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:34:28.457607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12f015-04a2-4381-a9e6-3ae80dccbde6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:28.478746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12f015-04a2-4381-a9e6-3ae80dccbde6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:28.484381 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746026e-afa7-4f69-9ec2-09527f35099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:28.485058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746026e-afa7-4f69-9ec2-09527f35099c admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:28.485335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746026e-afa7-4f69-9ec2-09527f35099c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:28.485619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746026e-afa7-4f69-9ec2-09527f35099c admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:28.486368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746026e-afa7-4f69-9ec2-09527f35099c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:28.488159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746026e-afa7-4f69-9ec2-09527f35099c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:28.499513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1131/2278] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:34:28.503680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e98265-21a9-45fb-91fa-417919a3a389 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:28.517255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e98265-21a9-45fb-91fa-417919a3a389 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:28.526423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1958a349-ffee-4051-bbc7-2683aac1b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:28.527259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958a349-ffee-4051-bbc7-2683aac1b3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:28.528228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958a349-ffee-4051-bbc7-2683aac1b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:28.528555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1958a349-ffee-4051-bbc7-2683aac1b3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:28.529365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958a349-ffee-4051-bbc7-2683aac1b3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:28.531072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1958a349-ffee-4051-bbc7-2683aac1b3d8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:28.541951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1148/2279] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:34:28.546374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746026e-afa7-4f69-9ec2-09527f35099c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:28.558137 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746026e-afa7-4f69-9ec2-09527f35099c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:28.563377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def10c71-d330-4312-82ca-a7ad88f1f7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:28.564283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def10c71-d330-4312-82ca-a7ad88f1f7f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:28.564572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def10c71-d330-4312-82ca-a7ad88f1f7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:28.564857 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def10c71-d330-4312-82ca-a7ad88f1f7f4 admin admin] PATH_INFO: `/v3/projects/afcb836a92c44fb6981fd88e4f0b8567/users/c47bfaf6d1d74efab6b3f731f4aa707e/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:28.583998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def10c71-d330-4312-82ca-a7ad88f1f7f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afcb836a92c44fb6981fd88e4f0b8567, user_id=c47bfaf6d1d74efab6b3f731f4aa707e, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:28.586812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def10c71-d330-4312-82ca-a7ad88f1f7f4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:28.605205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1132/2280] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:28 2022] PUT /identity/v3/projects/afcb836a92c44fb6981fd88e4f0b8567/users/c47bfaf6d1d74efab6b3f731f4aa707e/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:28.615752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fb5a4-648f-4515-9228-ad12726fb8f8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:28.616308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fb5a4-648f-4515-9228-ad12726fb8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:28.616592 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fb5a4-648f-4515-9228-ad12726fb8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:28.646363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-809fb5a4-648f-4515-9228-ad12726fb8f8 None None] MFA Rules not processed for user `c47bfaf6d1d74efab6b3f731f4aa707e`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:28.647652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-809fb5a4-648f-4515-9228-ad12726fb8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:28.735219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1149/2281] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:28 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:28.739655 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def10c71-d330-4312-82ca-a7ad88f1f7f4 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:28.807960 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def10c71-d330-4312-82ca-a7ad88f1f7f4 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:28.813571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23ada7-012c-48fc-8e26-f47c661c4f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:28.814234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23ada7-012c-48fc-8e26-f47c661c4f7a admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:28.814516 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23ada7-012c-48fc-8e26-f47c661c4f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:28.814819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23ada7-012c-48fc-8e26-f47c661c4f7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:28.815514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23ada7-012c-48fc-8e26-f47c661c4f7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:28.817419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23ada7-012c-48fc-8e26-f47c661c4f7a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:28.843788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1133/2282] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:34:28 2022] POST /identity/v3/projects => generated 357 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:28.848446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fb5a4-648f-4515-9228-ad12726fb8f8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:28.856039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:28.860612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fb5a4-648f-4515-9228-ad12726fb8f8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:28.943175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5501a-a93e-4052-8ebc-9cdf4ad950f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:28.944096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5501a-a93e-4052-8ebc-9cdf4ad950f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:28.944385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5501a-a93e-4052-8ebc-9cdf4ad950f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:28.944668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5501a-a93e-4052-8ebc-9cdf4ad950f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:28.945392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5501a-a93e-4052-8ebc-9cdf4ad950f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:28.947430 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5501a-a93e-4052-8ebc-9cdf4ad950f8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:28.976411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1150/2283] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:34:28 2022] POST /identity/v3/users => generated 285 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:28.982829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23ada7-012c-48fc-8e26-f47c661c4f7a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:28.998919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23ada7-012c-48fc-8e26-f47c661c4f7a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:29.004987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c688a-7fe1-45c9-9f5e-9305f32add4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:29.005682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c688a-7fe1-45c9-9f5e-9305f32add4a admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:29.005972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c688a-7fe1-45c9-9f5e-9305f32add4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:29.006254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c688a-7fe1-45c9-9f5e-9305f32add4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:29.007370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c688a-7fe1-45c9-9f5e-9305f32add4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:29.009021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c688a-7fe1-45c9-9f5e-9305f32add4a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:29.023011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1134/2284] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:29.027067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5501a-a93e-4052-8ebc-9cdf4ad950f8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:29.039072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5501a-a93e-4052-8ebc-9cdf4ad950f8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:29.044206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79629e-dd46-4493-a4cf-d0f0dcdc3338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:29.045114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79629e-dd46-4493-a4cf-d0f0dcdc3338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:29.045408 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79629e-dd46-4493-a4cf-d0f0dcdc3338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:29.045695 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79629e-dd46-4493-a4cf-d0f0dcdc3338 admin admin] PATH_INFO: `/v3/projects/9a7112ba1d2d409e9f4a0ddd81342085/users/9b33c8227f6b4ddaaa0597f5fed018a1/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:29.068175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79629e-dd46-4493-a4cf-d0f0dcdc3338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7112ba1d2d409e9f4a0ddd81342085, user_id=9b33c8227f6b4ddaaa0597f5fed018a1, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:29.071171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79629e-dd46-4493-a4cf-d0f0dcdc3338 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:29.092749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1151/2285] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:29 2022] PUT /identity/v3/projects/9a7112ba1d2d409e9f4a0ddd81342085/users/9b33c8227f6b4ddaaa0597f5fed018a1/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:29.097054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c688a-7fe1-45c9-9f5e-9305f32add4a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:29.108423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c688a-7fe1-45c9-9f5e-9305f32add4a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:29.156296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:29.177486 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f29ca-75f4-4d91-9295-77b3143c2f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:29.178403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f29ca-75f4-4d91-9295-77b3143c2f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:29.178735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f29ca-75f4-4d91-9295-77b3143c2f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:29.179020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f29ca-75f4-4d91-9295-77b3143c2f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:29.179909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f29ca-75f4-4d91-9295-77b3143c2f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:29.181608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f29ca-75f4-4d91-9295-77b3143c2f18 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:29.193474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1135/2286] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:29.197891 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79629e-dd46-4493-a4cf-d0f0dcdc3338 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:29.212906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79629e-dd46-4493-a4cf-d0f0dcdc3338 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:29.219505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862e072-b84b-4c0d-a92c-c3e7259d211a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:29.220514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862e072-b84b-4c0d-a92c-c3e7259d211a admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:29.220905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862e072-b84b-4c0d-a92c-c3e7259d211a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:29.221057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862e072-b84b-4c0d-a92c-c3e7259d211a admin admin] PATH_INFO: `/v3/projects/9a7112ba1d2d409e9f4a0ddd81342085/users/9b33c8227f6b4ddaaa0597f5fed018a1/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:29.223322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862e072-b84b-4c0d-a92c-c3e7259d211a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7112ba1d2d409e9f4a0ddd81342085, user_id=9b33c8227f6b4ddaaa0597f5fed018a1, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:29.227612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862e072-b84b-4c0d-a92c-c3e7259d211a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:29.241982 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1152/2287] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:29 2022] PUT /identity/v3/projects/9a7112ba1d2d409e9f4a0ddd81342085/users/9b33c8227f6b4ddaaa0597f5fed018a1/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:29.260061 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cdf0e-8e7f-4752-8d18-4a811c26ef38 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:29.260391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cdf0e-8e7f-4752-8d18-4a811c26ef38 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:29.260664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cdf0e-8e7f-4752-8d18-4a811c26ef38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:29.292233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-416cdf0e-8e7f-4752-8d18-4a811c26ef38 None None] MFA Rules not processed for user `9b33c8227f6b4ddaaa0597f5fed018a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:29.294041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-416cdf0e-8e7f-4752-8d18-4a811c26ef38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:29.365999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1136/2288] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:29 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:29.379827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862e072-b84b-4c0d-a92c-c3e7259d211a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:29.381538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1862e072-b84b-4c0d-a92c-c3e7259d211a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:29.473940 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862e072-b84b-4c0d-a92c-c3e7259d211a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:29.477752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e2abe-5ac9-4c38-b4a5-28b07cdd9ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:29.478398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e2abe-5ac9-4c38-b4a5-28b07cdd9ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:29.478713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e2abe-5ac9-4c38-b4a5-28b07cdd9ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:29.478990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e2abe-5ac9-4c38-b4a5-28b07cdd9ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:29.489530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e2abe-5ac9-4c38-b4a5-28b07cdd9ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:29.490966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e2abe-5ac9-4c38-b4a5-28b07cdd9ac5 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:29.501853 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1153/2289] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:29 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:34:29.578899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cdf0e-8e7f-4752-8d18-4a811c26ef38 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:29.588682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cdf0e-8e7f-4752-8d18-4a811c26ef38 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:29.592345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d55635-958e-4177-bb96-51b4fc2094a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:29.592995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55635-958e-4177-bb96-51b4fc2094a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:29.593278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55635-958e-4177-bb96-51b4fc2094a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:29.593551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d55635-958e-4177-bb96-51b4fc2094a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:29.658319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d55635-958e-4177-bb96-51b4fc2094a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:29.659882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d55635-958e-4177-bb96-51b4fc2094a1 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:29.670735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1137/2290] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:29 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:34:39.497038 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:39.665083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=71 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:46.263564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e2abe-5ac9-4c38-b4a5-28b07cdd9ac5 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:46.353915 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e2abe-5ac9-4c38-b4a5-28b07cdd9ac5 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:46.360637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:46.362351 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:46.362669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:46.362952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] PATH_INFO: `/v3/users/c47bfaf6d1d74efab6b3f731f4aa707e` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:46.364235 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c47bfaf6d1d74efab6b3f731f4aa707e)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:46.366343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:46.428160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c47bfaf6d1d74efab6b3f731f4aa707e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:46.443162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c47bfaf6d1d74efab6b3f731f4aa707e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:46.443896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c47bfaf6d1d74efab6b3f731f4aa707e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:46.452361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c47bfaf6d1d74efab6b3f731f4aa707e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:46.466475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c47bfaf6d1d74efab6b3f731f4aa707e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:46.476929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1154/2291] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:34:46 2022] DELETE /identity/v3/users/c47bfaf6d1d74efab6b3f731f4aa707e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:46.479139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f27e1e-8baa-4a78-acd4-d2dcfe00526d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:46.480178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f27e1e-8baa-4a78-acd4-d2dcfe00526d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:46.480439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f27e1e-8baa-4a78-acd4-d2dcfe00526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:46.499782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-81f27e1e-8baa-4a78-acd4-d2dcfe00526d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:46.501000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-81f27e1e-8baa-4a78-acd4-d2dcfe00526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:46.575048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1138/2292] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:46.586923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:46.588697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:46.679871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811f0d86-2464-4072-87c0-3d1add1e5816 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:46.686422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2d9e8-86cb-4aae-9270-4f79ebcff5ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:46.687128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2d9e8-86cb-4aae-9270-4f79ebcff5ab service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:46.687414 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2d9e8-86cb-4aae-9270-4f79ebcff5ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:46.687711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2d9e8-86cb-4aae-9270-4f79ebcff5ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:46.702969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2d9e8-86cb-4aae-9270-4f79ebcff5ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:46.704899 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2d9e8-86cb-4aae-9270-4f79ebcff5ab service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:46.717226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1155/2293] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:34:47.030404 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f27e1e-8baa-4a78-acd4-d2dcfe00526d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:47.031953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-81f27e1e-8baa-4a78-acd4-d2dcfe00526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:47.052190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f27e1e-8baa-4a78-acd4-d2dcfe00526d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:47.060227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:47.062671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:47.062972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:47.063396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] PATH_INFO: `/v3/projects/afcb836a92c44fb6981fd88e4f0b8567` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:47.065267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] RBAC: Authorizing `identity:delete_project(project_id=afcb836a92c44fb6981fd88e4f0b8567)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:47.067192 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:47.123156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afcb836a92c44fb6981fd88e4f0b8567', 'request_id': 'req-fa60aa2c-3d46-4e53-a48d-d2573a17460c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.129726 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afcb836a92c44fb6981fd88e4f0b8567', 'request_id': 'req-fa60aa2c-3d46-4e53-a48d-d2573a17460c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.138035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afcb836a92c44fb6981fd88e4f0b8567', 'request_id': 'req-fa60aa2c-3d46-4e53-a48d-d2573a17460c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.152133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] The token cache is being invalidate because project afcb836a92c44fb6981fd88e4f0b8567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:34:47.152418 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.153057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.154993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1139/2294] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:34:47 2022] DELETE /identity/v3/projects/afcb836a92c44fb6981fd88e4f0b8567 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:34:47.158858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2d9e8-86cb-4aae-9270-4f79ebcff5ab service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:47.170222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2d9e8-86cb-4aae-9270-4f79ebcff5ab service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:47.178769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:47.236139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f64815-df2d-4088-ab40-2c9cfdf8258d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:47.237055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f64815-df2d-4088-ab40-2c9cfdf8258d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:47.237366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f64815-df2d-4088-ab40-2c9cfdf8258d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:47.237645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f64815-df2d-4088-ab40-2c9cfdf8258d admin admin] PATH_INFO: `/v3/users/9b33c8227f6b4ddaaa0597f5fed018a1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:47.239012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f64815-df2d-4088-ab40-2c9cfdf8258d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b33c8227f6b4ddaaa0597f5fed018a1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:47.241073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f64815-df2d-4088-ab40-2c9cfdf8258d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:47.297272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b7f64815-df2d-4088-ab40-2c9cfdf8258d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b33c8227f6b4ddaaa0597f5fed018a1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.309977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b7f64815-df2d-4088-ab40-2c9cfdf8258d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b33c8227f6b4ddaaa0597f5fed018a1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.310811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b7f64815-df2d-4088-ab40-2c9cfdf8258d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b33c8227f6b4ddaaa0597f5fed018a1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.318805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b7f64815-df2d-4088-ab40-2c9cfdf8258d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b33c8227f6b4ddaaa0597f5fed018a1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.326420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b7f64815-df2d-4088-ab40-2c9cfdf8258d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b33c8227f6b4ddaaa0597f5fed018a1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.332420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1156/2295] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:34:47 2022] DELETE /identity/v3/users/9b33c8227f6b4ddaaa0597f5fed018a1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:47.609343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:47.611272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:47.626365 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa60aa2c-3d46-4e53-a48d-d2573a17460c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:47.685970 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:47.686763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:47.687054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:47.687336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] PATH_INFO: `/v3/projects/9a7112ba1d2d409e9f4a0ddd81342085` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:47.688537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a7112ba1d2d409e9f4a0ddd81342085)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:47.690458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:47.721746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:47.740739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a7112ba1d2d409e9f4a0ddd81342085', 'request_id': 'req-c49747b6-231a-4b93-86e2-4dc481bd8518'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.746273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a7112ba1d2d409e9f4a0ddd81342085', 'request_id': 'req-c49747b6-231a-4b93-86e2-4dc481bd8518'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.752485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a7112ba1d2d409e9f4a0ddd81342085', 'request_id': 'req-c49747b6-231a-4b93-86e2-4dc481bd8518'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.765898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] The token cache is being invalidate because project 9a7112ba1d2d409e9f4a0ddd81342085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:34:47.766190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.766850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c49747b6-231a-4b93-86e2-4dc481bd8518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:47.768383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1140/2296] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:34:47 2022] DELETE /identity/v3/projects/9a7112ba1d2d409e9f4a0ddd81342085 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:47.773613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50b6b2-4242-4f20-8621-b2f82710f0a6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:47.773806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50b6b2-4242-4f20-8621-b2f82710f0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:47.774085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50b6b2-4242-4f20-8621-b2f82710f0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:47.793533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3a50b6b2-4242-4f20-8621-b2f82710f0a6 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:47.794777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3a50b6b2-4242-4f20-8621-b2f82710f0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:47.857040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1157/2297] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:47.864977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baf67e-c2e2-4b57-8d4a-4ac42ad8ffb3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:47.865275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baf67e-c2e2-4b57-8d4a-4ac42ad8ffb3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:47.865561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baf67e-c2e2-4b57-8d4a-4ac42ad8ffb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:47.883661 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b4baf67e-c2e2-4b57-8d4a-4ac42ad8ffb3 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:47.884872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b4baf67e-c2e2-4b57-8d4a-4ac42ad8ffb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:47.893236 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1141/2298] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:47.897491 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50b6b2-4242-4f20-8621-b2f82710f0a6 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:47.908063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50b6b2-4242-4f20-8621-b2f82710f0a6 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:47.912826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0034bc-a4dd-46fb-bd89-1d5bc4416ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:47.913482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0034bc-a4dd-46fb-bd89-1d5bc4416ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:47.913759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0034bc-a4dd-46fb-bd89-1d5bc4416ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:47.914032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0034bc-a4dd-46fb-bd89-1d5bc4416ef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:47.914791 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0034bc-a4dd-46fb-bd89-1d5bc4416ef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:34:47.915095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0034bc-a4dd-46fb-bd89-1d5bc4416ef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:47.916708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0034bc-a4dd-46fb-bd89-1d5bc4416ef2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:47.929171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1158/2299] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:34:47.934389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baf67e-c2e2-4b57-8d4a-4ac42ad8ffb3 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:47.943944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baf67e-c2e2-4b57-8d4a-4ac42ad8ffb3 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:47.949051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496bcc7-bf2b-4752-8e56-d6d382625f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:47.949711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496bcc7-bf2b-4752-8e56-d6d382625f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:47.949996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496bcc7-bf2b-4752-8e56-d6d382625f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:47.950276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496bcc7-bf2b-4752-8e56-d6d382625f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:47.951012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496bcc7-bf2b-4752-8e56-d6d382625f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:47.952999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496bcc7-bf2b-4752-8e56-d6d382625f62 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:47.978779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1142/2300] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:34:47 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:47.984093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0034bc-a4dd-46fb-bd89-1d5bc4416ef2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:47.995498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0034bc-a4dd-46fb-bd89-1d5bc4416ef2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:48.015157 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:48.057191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12101159-cb6e-426b-8422-355e0826dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:48.057893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12101159-cb6e-426b-8422-355e0826dbab admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.058153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12101159-cb6e-426b-8422-355e0826dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.058670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12101159-cb6e-426b-8422-355e0826dbab admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.059375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12101159-cb6e-426b-8422-355e0826dbab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:48.061338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12101159-cb6e-426b-8422-355e0826dbab admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:48.083222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1159/2301] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:34:47 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:48.087656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496bcc7-bf2b-4752-8e56-d6d382625f62 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:48.098998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496bcc7-bf2b-4752-8e56-d6d382625f62 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:48.103661 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01aa677-ff7c-4363-9f63-41f546df77f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:48.104330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01aa677-ff7c-4363-9f63-41f546df77f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.104681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01aa677-ff7c-4363-9f63-41f546df77f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.105010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01aa677-ff7c-4363-9f63-41f546df77f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.105751 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01aa677-ff7c-4363-9f63-41f546df77f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:48.107529 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01aa677-ff7c-4363-9f63-41f546df77f4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:48.121439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1143/2302] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:48.125485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12101159-cb6e-426b-8422-355e0826dbab admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:48.137785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12101159-cb6e-426b-8422-355e0826dbab admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:48.144634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc603225-ad9d-4396-a534-84bcb423fd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:48.146005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc603225-ad9d-4396-a534-84bcb423fd2f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.146354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc603225-ad9d-4396-a534-84bcb423fd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.146720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc603225-ad9d-4396-a534-84bcb423fd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.147418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc603225-ad9d-4396-a534-84bcb423fd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:48.149122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc603225-ad9d-4396-a534-84bcb423fd2f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:48.162276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1160/2303] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:48.165383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01aa677-ff7c-4363-9f63-41f546df77f4 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:48.176667 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01aa677-ff7c-4363-9f63-41f546df77f4 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:48.181578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93299ff0-f737-41d8-ada2-dbbe6b5dd678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:48.182533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93299ff0-f737-41d8-ada2-dbbe6b5dd678 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.182883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93299ff0-f737-41d8-ada2-dbbe6b5dd678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.183161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93299ff0-f737-41d8-ada2-dbbe6b5dd678 admin admin] PATH_INFO: `/v3/projects/753b06c70e4a4c86bddb9beccdb0b4bc/users/e3cd38c790094273ab4c613ec63cc858/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.200754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93299ff0-f737-41d8-ada2-dbbe6b5dd678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=753b06c70e4a4c86bddb9beccdb0b4bc, user_id=e3cd38c790094273ab4c613ec63cc858, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:48.203455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93299ff0-f737-41d8-ada2-dbbe6b5dd678 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:48.219566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1144/2304] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:48 2022] PUT /identity/v3/projects/753b06c70e4a4c86bddb9beccdb0b4bc/users/e3cd38c790094273ab4c613ec63cc858/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:34:48.230847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aa517-8060-4e97-a274-023bdcb8651f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.231432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aa517-8060-4e97-a274-023bdcb8651f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.231757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3aa517-8060-4e97-a274-023bdcb8651f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.261577 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4f3aa517-8060-4e97-a274-023bdcb8651f None None] MFA Rules not processed for user `e3cd38c790094273ab4c613ec63cc858`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:48.263064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4f3aa517-8060-4e97-a274-023bdcb8651f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:48.337574 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1161/2305] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:48.342889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93299ff0-f737-41d8-ada2-dbbe6b5dd678 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:48.410435 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93299ff0-f737-41d8-ada2-dbbe6b5dd678 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:48.415399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b88cf2-b78e-467d-af00-893404e10f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:48.416109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b88cf2-b78e-467d-af00-893404e10f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.416382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b88cf2-b78e-467d-af00-893404e10f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.416660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b88cf2-b78e-467d-af00-893404e10f55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.417492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b88cf2-b78e-467d-af00-893404e10f55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:48.419477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b88cf2-b78e-467d-af00-893404e10f55 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:48.446271 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1145/2306] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:34:48 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:34:48.451233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3aa517-8060-4e97-a274-023bdcb8651f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:48.462400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3aa517-8060-4e97-a274-023bdcb8651f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:48.519890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd0c0d-cbf6-4d21-ab96-4811ccd19b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:48.520562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd0c0d-cbf6-4d21-ab96-4811ccd19b40 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.520847 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd0c0d-cbf6-4d21-ab96-4811ccd19b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.521121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd0c0d-cbf6-4d21-ab96-4811ccd19b40 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.521810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd0c0d-cbf6-4d21-ab96-4811ccd19b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:48.523784 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd0c0d-cbf6-4d21-ab96-4811ccd19b40 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:48.543889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1162/2307] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:34:48 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:48.548105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b88cf2-b78e-467d-af00-893404e10f55 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:48.559729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b88cf2-b78e-467d-af00-893404e10f55 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:48.564377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f3df9-2965-470a-9e12-7d6b9bc6c944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:48.565040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f3df9-2965-470a-9e12-7d6b9bc6c944 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.565327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f3df9-2965-470a-9e12-7d6b9bc6c944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.565608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f3df9-2965-470a-9e12-7d6b9bc6c944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.566344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f3df9-2965-470a-9e12-7d6b9bc6c944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:48.567993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f3df9-2965-470a-9e12-7d6b9bc6c944 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:48.580321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1146/2308] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:48.584470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd0c0d-cbf6-4d21-ab96-4811ccd19b40 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:48.595020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd0c0d-cbf6-4d21-ab96-4811ccd19b40 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:48.600068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48996761-be2e-4906-a084-64bc1f7ea279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:48.600942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48996761-be2e-4906-a084-64bc1f7ea279 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.601236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48996761-be2e-4906-a084-64bc1f7ea279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.601505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48996761-be2e-4906-a084-64bc1f7ea279 admin admin] PATH_INFO: `/v3/projects/b5d7cf334dee4c65a292a06ff0798646/users/d1dfda701d5541d485b6f1c144fe7a65/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.613955 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:48.617393 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48996761-be2e-4906-a084-64bc1f7ea279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d7cf334dee4c65a292a06ff0798646, user_id=d1dfda701d5541d485b6f1c144fe7a65, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:48.620836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48996761-be2e-4906-a084-64bc1f7ea279 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:48.635295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1163/2309] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:48 2022] PUT /identity/v3/projects/b5d7cf334dee4c65a292a06ff0798646/users/d1dfda701d5541d485b6f1c144fe7a65/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:48.640216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f3df9-2965-470a-9e12-7d6b9bc6c944 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:48.650135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f3df9-2965-470a-9e12-7d6b9bc6c944 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:48.677497 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:48.720249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee7d42-8905-4c3d-b339-3ab0ec83dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:48.720828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee7d42-8905-4c3d-b339-3ab0ec83dd4f admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.721111 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee7d42-8905-4c3d-b339-3ab0ec83dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.721385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee7d42-8905-4c3d-b339-3ab0ec83dd4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.722111 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee7d42-8905-4c3d-b339-3ab0ec83dd4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:48.723739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee7d42-8905-4c3d-b339-3ab0ec83dd4f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:48.734322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1147/2310] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:34:48.738503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48996761-be2e-4906-a084-64bc1f7ea279 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:48.750269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48996761-be2e-4906-a084-64bc1f7ea279 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:48.755197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2230147-cb4e-44e5-bb55-98cd5218c7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:48.755843 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2230147-cb4e-44e5-bb55-98cd5218c7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.756132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2230147-cb4e-44e5-bb55-98cd5218c7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.756417 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2230147-cb4e-44e5-bb55-98cd5218c7ac admin admin] PATH_INFO: `/v3/projects/b5d7cf334dee4c65a292a06ff0798646/users/d1dfda701d5541d485b6f1c144fe7a65/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.758513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2230147-cb4e-44e5-bb55-98cd5218c7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d7cf334dee4c65a292a06ff0798646, user_id=d1dfda701d5541d485b6f1c144fe7a65, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:48.761298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2230147-cb4e-44e5-bb55-98cd5218c7ac admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:48.772768 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1164/2311] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:48 2022] PUT /identity/v3/projects/b5d7cf334dee4c65a292a06ff0798646/users/d1dfda701d5541d485b6f1c144fe7a65/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:48.784061 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57ec36-ced1-465a-ac6f-489d53249ef1 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.784430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57ec36-ced1-465a-ac6f-489d53249ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.784818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb57ec36-ced1-465a-ac6f-489d53249ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.812305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-eb57ec36-ced1-465a-ac6f-489d53249ef1 None None] MFA Rules not processed for user `d1dfda701d5541d485b6f1c144fe7a65`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:48.813533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-eb57ec36-ced1-465a-ac6f-489d53249ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:48.882016 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1148/2312] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:48 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:48.893954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2230147-cb4e-44e5-bb55-98cd5218c7ac admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:48.895726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d2230147-cb4e-44e5-bb55-98cd5218c7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:48.955574 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2230147-cb4e-44e5-bb55-98cd5218c7ac admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:48.959320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08000279-d3e1-480c-8910-7ec3c38fe144 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:48.959967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000279-d3e1-480c-8910-7ec3c38fe144 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:48.960249 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000279-d3e1-480c-8910-7ec3c38fe144 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:48.960525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08000279-d3e1-480c-8910-7ec3c38fe144 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:48.969897 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000279-d3e1-480c-8910-7ec3c38fe144 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:48.971432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08000279-d3e1-480c-8910-7ec3c38fe144 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:48.981423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1165/2313] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:48 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:34:49.027566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57ec36-ced1-465a-ac6f-489d53249ef1 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:49.028986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-eb57ec36-ced1-465a-ac6f-489d53249ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:49.040925 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb57ec36-ced1-465a-ac6f-489d53249ef1 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:49.045107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0843a3-f43a-49d3-84ce-663aeb2d4236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:49.045755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0843a3-f43a-49d3-84ce-663aeb2d4236 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:49.046045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0843a3-f43a-49d3-84ce-663aeb2d4236 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:49.046325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0843a3-f43a-49d3-84ce-663aeb2d4236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:49.103413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0843a3-f43a-49d3-84ce-663aeb2d4236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:49.105547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0843a3-f43a-49d3-84ce-663aeb2d4236 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:49.114524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1149/2314] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:34:52.536921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08000279-d3e1-480c-8910-7ec3c38fe144 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:52.601732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08000279-d3e1-480c-8910-7ec3c38fe144 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:52.608717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:52.609607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:52.609907 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:52.610199 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] PATH_INFO: `/v3/users/e3cd38c790094273ab4c613ec63cc858` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:52.611454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3cd38c790094273ab4c613ec63cc858)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:52.613419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:52.665954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3cd38c790094273ab4c613ec63cc858'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:52.678968 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3cd38c790094273ab4c613ec63cc858'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:52.679690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3cd38c790094273ab4c613ec63cc858'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:52.686881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3cd38c790094273ab4c613ec63cc858'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:52.694176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3cd38c790094273ab4c613ec63cc858'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:52.696404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:52.702293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1166/2315] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:34:52 2022] DELETE /identity/v3/users/e3cd38c790094273ab4c613ec63cc858 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:52.706975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82facc88-d933-4039-b136-a31624e69433 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:52.707449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82facc88-d933-4039-b136-a31624e69433 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:52.707577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82facc88-d933-4039-b136-a31624e69433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:52.726016 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-82facc88-d933-4039-b136-a31624e69433 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:52.727266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-82facc88-d933-4039-b136-a31624e69433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:52.768105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:52.787850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1150/2316] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:52.797452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:52.798864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:52.856697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c265c5-ae43-4d66-8b53-bbf08722ec93 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:52.860336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa2eb8-5efd-4c78-a2b6-d5768a62ac8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:52.860980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa2eb8-5efd-4c78-a2b6-d5768a62ac8c service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:52.861266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa2eb8-5efd-4c78-a2b6-d5768a62ac8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:52.861541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa2eb8-5efd-4c78-a2b6-d5768a62ac8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:52.871056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa2eb8-5efd-4c78-a2b6-d5768a62ac8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:52.872494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa2eb8-5efd-4c78-a2b6-d5768a62ac8c service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:52.882238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1167/2317] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:34:53.218655 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82facc88-d933-4039-b136-a31624e69433 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:53.220283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-82facc88-d933-4039-b136-a31624e69433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:53.240771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82facc88-d933-4039-b136-a31624e69433 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:53.247441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:53.248167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:53.248453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:53.248737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] PATH_INFO: `/v3/projects/753b06c70e4a4c86bddb9beccdb0b4bc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:53.250504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=753b06c70e4a4c86bddb9beccdb0b4bc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:53.257480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:53.322202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '753b06c70e4a4c86bddb9beccdb0b4bc', 'request_id': 'req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.328786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '753b06c70e4a4c86bddb9beccdb0b4bc', 'request_id': 'req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.335146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '753b06c70e4a4c86bddb9beccdb0b4bc', 'request_id': 'req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.347917 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] The token cache is being invalidate because project 753b06c70e4a4c86bddb9beccdb0b4bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:34:53.348215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.348850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.350400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1151/2318] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:34:53 2022] DELETE /identity/v3/projects/753b06c70e4a4c86bddb9beccdb0b4bc => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:34:53.354984 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa2eb8-5efd-4c78-a2b6-d5768a62ac8c service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:53.364723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa2eb8-5efd-4c78-a2b6-d5768a62ac8c service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:53.426768 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12169a1-c8d4-4a45-88f8-d119b3d10a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:53.427668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12169a1-c8d4-4a45-88f8-d119b3d10a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:53.427951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12169a1-c8d4-4a45-88f8-d119b3d10a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:53.428230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12169a1-c8d4-4a45-88f8-d119b3d10a51 admin admin] PATH_INFO: `/v3/users/d1dfda701d5541d485b6f1c144fe7a65` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:53.429477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12169a1-c8d4-4a45-88f8-d119b3d10a51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1dfda701d5541d485b6f1c144fe7a65)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:53.431500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12169a1-c8d4-4a45-88f8-d119b3d10a51 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:53.490844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e12169a1-c8d4-4a45-88f8-d119b3d10a51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1dfda701d5541d485b6f1c144fe7a65'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.505409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e12169a1-c8d4-4a45-88f8-d119b3d10a51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1dfda701d5541d485b6f1c144fe7a65'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.506147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e12169a1-c8d4-4a45-88f8-d119b3d10a51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1dfda701d5541d485b6f1c144fe7a65'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.514909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e12169a1-c8d4-4a45-88f8-d119b3d10a51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1dfda701d5541d485b6f1c144fe7a65'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.522917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e12169a1-c8d4-4a45-88f8-d119b3d10a51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1dfda701d5541d485b6f1c144fe7a65'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.532542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1168/2319] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:34:53 2022] DELETE /identity/v3/users/d1dfda701d5541d485b6f1c144fe7a65 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:53.828912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:53.830603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:53.845720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839bb7f4-553d-48dd-b752-1c8b0ccd2d2c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:53.917760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:53.918565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:53.918882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:53.919160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] PATH_INFO: `/v3/projects/b5d7cf334dee4c65a292a06ff0798646` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:53.920464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5d7cf334dee4c65a292a06ff0798646)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:53.922449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:53.983140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5d7cf334dee4c65a292a06ff0798646', 'request_id': 'req-07ae4ff6-d1f1-4ba3-9af3-c09184721358'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.989091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5d7cf334dee4c65a292a06ff0798646', 'request_id': 'req-07ae4ff6-d1f1-4ba3-9af3-c09184721358'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:53.996038 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5d7cf334dee4c65a292a06ff0798646', 'request_id': 'req-07ae4ff6-d1f1-4ba3-9af3-c09184721358'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:54.005942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:54.012496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] The token cache is being invalidate because project b5d7cf334dee4c65a292a06ff0798646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:34:54.012804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:54.013521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-07ae4ff6-d1f1-4ba3-9af3-c09184721358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:54.015666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1152/2320] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:34:53 2022] DELETE /identity/v3/projects/b5d7cf334dee4c65a292a06ff0798646 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:34:54.021062 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01c25a-48e3-45d9-998c-149df69617f2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.021603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01c25a-48e3-45d9-998c-149df69617f2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.021603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01c25a-48e3-45d9-998c-149df69617f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.041258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-dd01c25a-48e3-45d9-998c-149df69617f2 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:54.042541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dd01c25a-48e3-45d9-998c-149df69617f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:54.059871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:54.105092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1169/2321] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:54.113133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575b8af-1864-49c9-888e-9c0937bd0434 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.113425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575b8af-1864-49c9-888e-9c0937bd0434 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.113730 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575b8af-1864-49c9-888e-9c0937bd0434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.131979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6575b8af-1864-49c9-888e-9c0937bd0434 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:54.133184 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6575b8af-1864-49c9-888e-9c0937bd0434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:54.142252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1153/2322] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:54.146444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01c25a-48e3-45d9-998c-149df69617f2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:54.156781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01c25a-48e3-45d9-998c-149df69617f2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:54.161841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec477c9-0ccc-427b-84eb-39ffb5567592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:54.162497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec477c9-0ccc-427b-84eb-39ffb5567592 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.162812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec477c9-0ccc-427b-84eb-39ffb5567592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.163093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec477c9-0ccc-427b-84eb-39ffb5567592 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.163823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec477c9-0ccc-427b-84eb-39ffb5567592 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:34:54.164132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec477c9-0ccc-427b-84eb-39ffb5567592 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:54.165758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec477c9-0ccc-427b-84eb-39ffb5567592 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:54.178324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1170/2323] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:34:54.183344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575b8af-1864-49c9-888e-9c0937bd0434 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:54.192612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575b8af-1864-49c9-888e-9c0937bd0434 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:54.197327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68624f6f-56f5-49e0-afd7-e1e6ea38ecc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:54.198051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68624f6f-56f5-49e0-afd7-e1e6ea38ecc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.198346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68624f6f-56f5-49e0-afd7-e1e6ea38ecc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.198639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68624f6f-56f5-49e0-afd7-e1e6ea38ecc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.199360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68624f6f-56f5-49e0-afd7-e1e6ea38ecc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:54.201336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68624f6f-56f5-49e0-afd7-e1e6ea38ecc9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:54.231621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1154/2324] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:34:54 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:34:54.236385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec477c9-0ccc-427b-84eb-39ffb5567592 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:54.248308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec477c9-0ccc-427b-84eb-39ffb5567592 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:54.320077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3226d-8c62-44ac-841c-ed9b64b23994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:54.320770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3226d-8c62-44ac-841c-ed9b64b23994 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.321053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3226d-8c62-44ac-841c-ed9b64b23994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.321326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3226d-8c62-44ac-841c-ed9b64b23994 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.322032 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3226d-8c62-44ac-841c-ed9b64b23994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:54.324379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3226d-8c62-44ac-841c-ed9b64b23994 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:54.351596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1171/2325] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:34:54 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:54.356088 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68624f6f-56f5-49e0-afd7-e1e6ea38ecc9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:54.366852 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68624f6f-56f5-49e0-afd7-e1e6ea38ecc9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:54.372537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051b0b1-149a-4eef-9b92-ac762980f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:54.373220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051b0b1-149a-4eef-9b92-ac762980f619 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.373504 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051b0b1-149a-4eef-9b92-ac762980f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.374608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051b0b1-149a-4eef-9b92-ac762980f619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.374608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051b0b1-149a-4eef-9b92-ac762980f619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:54.376214 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051b0b1-149a-4eef-9b92-ac762980f619 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:54.391578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1155/2326] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:34:54.395723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3226d-8c62-44ac-841c-ed9b64b23994 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:54.407281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3226d-8c62-44ac-841c-ed9b64b23994 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:54.412643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd2b6d-cebd-4434-a24d-5a597018681f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:54.413320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd2b6d-cebd-4434-a24d-5a597018681f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.413610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd2b6d-cebd-4434-a24d-5a597018681f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.413889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd2b6d-cebd-4434-a24d-5a597018681f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.414647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd2b6d-cebd-4434-a24d-5a597018681f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:54.416281 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd2b6d-cebd-4434-a24d-5a597018681f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:54.429521 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1172/2327] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:54.433770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051b0b1-149a-4eef-9b92-ac762980f619 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:54.444300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051b0b1-149a-4eef-9b92-ac762980f619 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:54.449352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bc864-87be-42cf-b2ee-e694aecb8817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:54.450242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bc864-87be-42cf-b2ee-e694aecb8817 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.450530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bc864-87be-42cf-b2ee-e694aecb8817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.451092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bc864-87be-42cf-b2ee-e694aecb8817 admin admin] PATH_INFO: `/v3/projects/fc1baf867893461dbf25150566b9ff5d/users/6e66f10a2fe843b5a9f9db937bb1d900/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.472959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bc864-87be-42cf-b2ee-e694aecb8817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc1baf867893461dbf25150566b9ff5d, user_id=6e66f10a2fe843b5a9f9db937bb1d900, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:54.475823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bc864-87be-42cf-b2ee-e694aecb8817 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:54.492960 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1156/2328] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:54 2022] PUT /identity/v3/projects/fc1baf867893461dbf25150566b9ff5d/users/6e66f10a2fe843b5a9f9db937bb1d900/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:54.503950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb82e5-7a84-49ed-bf36-add255856fa4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.504242 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb82e5-7a84-49ed-bf36-add255856fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.504523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb82e5-7a84-49ed-bf36-add255856fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.536151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-cccb82e5-7a84-49ed-bf36-add255856fa4 None None] MFA Rules not processed for user `6e66f10a2fe843b5a9f9db937bb1d900`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:54.537405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cccb82e5-7a84-49ed-bf36-add255856fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:54.609367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1173/2329] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:54.613890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bc864-87be-42cf-b2ee-e694aecb8817 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:54.692400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bc864-87be-42cf-b2ee-e694aecb8817 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:54.698514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7349a59-bce7-4867-9c63-fcdc6a52b384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:54.699340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7349a59-bce7-4867-9c63-fcdc6a52b384 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.699632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7349a59-bce7-4867-9c63-fcdc6a52b384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.699914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7349a59-bce7-4867-9c63-fcdc6a52b384 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.700638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7349a59-bce7-4867-9c63-fcdc6a52b384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:54.702816 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7349a59-bce7-4867-9c63-fcdc6a52b384 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:54.738740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1157/2330] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:34:54 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:54.741045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb82e5-7a84-49ed-bf36-add255856fa4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:54.751583 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb82e5-7a84-49ed-bf36-add255856fa4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:54.767324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:54.817566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcdc11-454e-4009-b8bd-7a95fc46c983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:54.818263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcdc11-454e-4009-b8bd-7a95fc46c983 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.818546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcdc11-454e-4009-b8bd-7a95fc46c983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.818840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcdc11-454e-4009-b8bd-7a95fc46c983 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.819704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcdc11-454e-4009-b8bd-7a95fc46c983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:54.821653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcdc11-454e-4009-b8bd-7a95fc46c983 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:54.845238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1174/2331] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:34:54 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:54.849399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7349a59-bce7-4867-9c63-fcdc6a52b384 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:54.868280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7349a59-bce7-4867-9c63-fcdc6a52b384 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:54.873191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb12658-33af-4acc-ac98-b9894fc25eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:54.874744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb12658-33af-4acc-ac98-b9894fc25eef admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.874744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb12658-33af-4acc-ac98-b9894fc25eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.874744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb12658-33af-4acc-ac98-b9894fc25eef admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.875737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb12658-33af-4acc-ac98-b9894fc25eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:54.876848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb12658-33af-4acc-ac98-b9894fc25eef admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:54.889452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1158/2332] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:54.893697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcdc11-454e-4009-b8bd-7a95fc46c983 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:54.905693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcdc11-454e-4009-b8bd-7a95fc46c983 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:54.910434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302a30e-06ac-450e-b903-3bdfc235a3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:54.911381 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302a30e-06ac-450e-b903-3bdfc235a3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:54.911667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302a30e-06ac-450e-b903-3bdfc235a3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:54.911932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302a30e-06ac-450e-b903-3bdfc235a3a8 admin admin] PATH_INFO: `/v3/projects/9076da63c34c4d24b7651fb9a72d0975/users/02168d68f4954d53b3200a076828b9fa/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:54.929573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302a30e-06ac-450e-b903-3bdfc235a3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9076da63c34c4d24b7651fb9a72d0975, user_id=02168d68f4954d53b3200a076828b9fa, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:54.932324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302a30e-06ac-450e-b903-3bdfc235a3a8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:54.948567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1175/2333] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:54 2022] PUT /identity/v3/projects/9076da63c34c4d24b7651fb9a72d0975/users/02168d68f4954d53b3200a076828b9fa/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:34:54.952509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb12658-33af-4acc-ac98-b9894fc25eef admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:54.962808 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb12658-33af-4acc-ac98-b9894fc25eef admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:55.029447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa0ce6-b61a-4dc6-b1e6-3aeeaee6f243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:55.031304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa0ce6-b61a-4dc6-b1e6-3aeeaee6f243 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:55.031601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa0ce6-b61a-4dc6-b1e6-3aeeaee6f243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:55.031886 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa0ce6-b61a-4dc6-b1e6-3aeeaee6f243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:55.032663 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa0ce6-b61a-4dc6-b1e6-3aeeaee6f243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:55.034284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aa0ce6-b61a-4dc6-b1e6-3aeeaee6f243 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:55.045539 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1159/2334] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:55.049593 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302a30e-06ac-450e-b903-3bdfc235a3a8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:55.061092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302a30e-06ac-450e-b903-3bdfc235a3a8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:55.066435 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f3996-5f11-4b70-be9d-b9538bddee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:55.067364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3996-5f11-4b70-be9d-b9538bddee1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:55.067669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3996-5f11-4b70-be9d-b9538bddee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:55.067942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f3996-5f11-4b70-be9d-b9538bddee1f admin admin] PATH_INFO: `/v3/projects/9076da63c34c4d24b7651fb9a72d0975/users/02168d68f4954d53b3200a076828b9fa/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:55.070283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f3996-5f11-4b70-be9d-b9538bddee1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9076da63c34c4d24b7651fb9a72d0975, user_id=02168d68f4954d53b3200a076828b9fa, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:55.072619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f3996-5f11-4b70-be9d-b9538bddee1f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:55.084418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1176/2335] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:55 2022] PUT /identity/v3/projects/9076da63c34c4d24b7651fb9a72d0975/users/02168d68f4954d53b3200a076828b9fa/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:55.096742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe7b6e-5828-4fe7-8eab-6220c12300b4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:55.097145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe7b6e-5828-4fe7-8eab-6220c12300b4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:55.097318 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe7b6e-5828-4fe7-8eab-6220c12300b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:55.104607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:55.126984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-cdbe7b6e-5828-4fe7-8eab-6220c12300b4 None None] MFA Rules not processed for user `02168d68f4954d53b3200a076828b9fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:55.128207 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cdbe7b6e-5828-4fe7-8eab-6220c12300b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:55.199477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1160/2336] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:55.211667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f3996-5f11-4b70-be9d-b9538bddee1f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:55.213468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-293f3996-5f11-4b70-be9d-b9538bddee1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:55.278345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f3996-5f11-4b70-be9d-b9538bddee1f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:55.282075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8071420-3920-4583-b7ec-c8c1da636a1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:55.282773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8071420-3920-4583-b7ec-c8c1da636a1e service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:55.283055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8071420-3920-4583-b7ec-c8c1da636a1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:55.283333 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8071420-3920-4583-b7ec-c8c1da636a1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:55.293115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8071420-3920-4583-b7ec-c8c1da636a1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:55.294572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8071420-3920-4583-b7ec-c8c1da636a1e service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:55.304540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1177/2337] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:34:55.632012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe7b6e-5828-4fe7-8eab-6220c12300b4 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:55.643187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe7b6e-5828-4fe7-8eab-6220c12300b4 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:55.648104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15cc44-6909-4791-bfeb-68797cca718d tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02168d68f4954d53b3200a076828b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9076da63c34c4d24b7651fb9a72d0975', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:55.648848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15cc44-6909-4791-bfeb-68797cca718d tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:55.649230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15cc44-6909-4791-bfeb-68797cca718d tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:55.649590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15cc44-6909-4791-bfeb-68797cca718d tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:55.650498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15cc44-6909-4791-bfeb-68797cca718d tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:34:55.650940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15cc44-6909-4791-bfeb-68797cca718d tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:55.652680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f15cc44-6909-4791-bfeb-68797cca718d tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:55.664942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1161/2338] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:34:55.670874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8071420-3920-4583-b7ec-c8c1da636a1e service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:55.680658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8071420-3920-4583-b7ec-c8c1da636a1e service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:55.685947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697aeb2-45c2-40b8-b318-437d985db53e tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02168d68f4954d53b3200a076828b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9076da63c34c4d24b7651fb9a72d0975', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:55.686638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697aeb2-45c2-40b8-b318-437d985db53e tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:55.686931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697aeb2-45c2-40b8-b318-437d985db53e tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:55.687213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697aeb2-45c2-40b8-b318-437d985db53e tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:55.687936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697aeb2-45c2-40b8-b318-437d985db53e tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:55.689900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2697aeb2-45c2-40b8-b318-437d985db53e tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:55.716952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1178/2339] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:34:55 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:55.721352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15cc44-6909-4791-bfeb-68797cca718d tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:55.732815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15cc44-6909-4791-bfeb-68797cca718d tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:55.790842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae5879-d7d1-4792-afba-00c2dfaeac8b tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02168d68f4954d53b3200a076828b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9076da63c34c4d24b7651fb9a72d0975', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:55.792312 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae5879-d7d1-4792-afba-00c2dfaeac8b tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:55.792483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae5879-d7d1-4792-afba-00c2dfaeac8b tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:55.792823 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae5879-d7d1-4792-afba-00c2dfaeac8b tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:55.793776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae5879-d7d1-4792-afba-00c2dfaeac8b tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:34:55.794090 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae5879-d7d1-4792-afba-00c2dfaeac8b tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:55.795736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae5879-d7d1-4792-afba-00c2dfaeac8b tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:55.809711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1162/2340] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:34:55.846366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697aeb2-45c2-40b8-b318-437d985db53e tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:55.858072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697aeb2-45c2-40b8-b318-437d985db53e tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:55.863000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02168d68f4954d53b3200a076828b9fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9076da63c34c4d24b7651fb9a72d0975', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:55.864012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:55.864316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:55.864596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] PATH_INFO: `/v3/projects/3985fe908ac24bb3a64cd21d59c60bab` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:55.865873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3985fe908ac24bb3a64cd21d59c60bab)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:55.867840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:55.916320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:55.920569 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3985fe908ac24bb3a64cd21d59c60bab', 'request_id': 'req-bcbc948d-ce44-4ad7-84b3-9fa90661508a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:55.937814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3985fe908ac24bb3a64cd21d59c60bab', 'request_id': 'req-bcbc948d-ce44-4ad7-84b3-9fa90661508a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:55.944966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3985fe908ac24bb3a64cd21d59c60bab', 'request_id': 'req-bcbc948d-ce44-4ad7-84b3-9fa90661508a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:55.956762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] The token cache is being invalidate because project 3985fe908ac24bb3a64cd21d59c60bab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:34:55.957055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:55.957739 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-bcbc948d-ce44-4ad7-84b3-9fa90661508a tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:55.959367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1179/2341] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:34:55 2022] DELETE /identity/v3/projects/3985fe908ac24bb3a64cd21d59c60bab => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:56.171395 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae5879-d7d1-4792-afba-00c2dfaeac8b tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:56.173320 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2bae5879-d7d1-4792-afba-00c2dfaeac8b tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:56.242048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae5879-d7d1-4792-afba-00c2dfaeac8b tempest-VolumeQuotaClassesTest-1940610124 tempest-VolumeQuotaClassesTest-1940610124-project-admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:56.255685 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:56.305897 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:56.306835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:56.307129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:56.307575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] PATH_INFO: `/v3/users/6e66f10a2fe843b5a9f9db937bb1d900` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:56.308926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e66f10a2fe843b5a9f9db937bb1d900)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:56.310993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:56.363211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e66f10a2fe843b5a9f9db937bb1d900'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:56.370337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e66f10a2fe843b5a9f9db937bb1d900'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:56.378154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e66f10a2fe843b5a9f9db937bb1d900'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:56.391425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e66f10a2fe843b5a9f9db937bb1d900'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:56.392100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e66f10a2fe843b5a9f9db937bb1d900'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:56.398474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1163/2342] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:34:56 2022] DELETE /identity/v3/users/6e66f10a2fe843b5a9f9db937bb1d900 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:56.404139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243611b-eca7-4d92-9a16-9a90e4b4fdd0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:56.404678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243611b-eca7-4d92-9a16-9a90e4b4fdd0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:56.404678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243611b-eca7-4d92-9a16-9a90e4b4fdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:56.425285 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-4243611b-eca7-4d92-9a16-9a90e4b4fdd0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:56.429105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4243611b-eca7-4d92-9a16-9a90e4b4fdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:56.491772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1180/2343] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:34:56.501762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:56.503270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:56.571587 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761268b1-36b2-455d-9dba-6f0e813d36e7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:56.576639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b389d-94b9-4c92-9ab2-d973c56056d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:56.577313 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b389d-94b9-4c92-9ab2-d973c56056d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:56.577596 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b389d-94b9-4c92-9ab2-d973c56056d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:56.577876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b389d-94b9-4c92-9ab2-d973c56056d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:56.588071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b389d-94b9-4c92-9ab2-d973c56056d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:56.590662 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9b389d-94b9-4c92-9ab2-d973c56056d0 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:56.601443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1164/2344] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:34:56.957943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243611b-eca7-4d92-9a16-9a90e4b4fdd0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:56.959447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4243611b-eca7-4d92-9a16-9a90e4b4fdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:56.972709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243611b-eca7-4d92-9a16-9a90e4b4fdd0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:56.977789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:56.978824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:56.980003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:56.980372 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] PATH_INFO: `/v3/projects/fc1baf867893461dbf25150566b9ff5d` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:56.982146 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc1baf867893461dbf25150566b9ff5d)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:56.984975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:57.052156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc1baf867893461dbf25150566b9ff5d', 'request_id': 'req-779000c4-4534-4990-85fc-82c44a867dfe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.061474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc1baf867893461dbf25150566b9ff5d', 'request_id': 'req-779000c4-4534-4990-85fc-82c44a867dfe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.073123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc1baf867893461dbf25150566b9ff5d', 'request_id': 'req-779000c4-4534-4990-85fc-82c44a867dfe'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.087399 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] The token cache is being invalidate because project fc1baf867893461dbf25150566b9ff5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:34:57.087740 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.088601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.090478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1181/2345] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:34:56 2022] DELETE /identity/v3/projects/fc1baf867893461dbf25150566b9ff5d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:57.094594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b389d-94b9-4c92-9ab2-d973c56056d0 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:57.106522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b389d-94b9-4c92-9ab2-d973c56056d0 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:57.151481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:57.173464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c475445-433d-42b5-aaa0-d593393c94d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:57.174396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c475445-433d-42b5-aaa0-d593393c94d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:57.174720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c475445-433d-42b5-aaa0-d593393c94d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:57.174995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c475445-433d-42b5-aaa0-d593393c94d8 admin admin] PATH_INFO: `/v3/users/02168d68f4954d53b3200a076828b9fa` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:57.176214 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c475445-433d-42b5-aaa0-d593393c94d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02168d68f4954d53b3200a076828b9fa)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:57.178151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c475445-433d-42b5-aaa0-d593393c94d8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:57.234712 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8c475445-433d-42b5-aaa0-d593393c94d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02168d68f4954d53b3200a076828b9fa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.243119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8c475445-433d-42b5-aaa0-d593393c94d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02168d68f4954d53b3200a076828b9fa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.252653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8c475445-433d-42b5-aaa0-d593393c94d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02168d68f4954d53b3200a076828b9fa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.266916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8c475445-433d-42b5-aaa0-d593393c94d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02168d68f4954d53b3200a076828b9fa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.267646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8c475445-433d-42b5-aaa0-d593393c94d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02168d68f4954d53b3200a076828b9fa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.278507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1165/2346] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:34:57 2022] DELETE /identity/v3/users/02168d68f4954d53b3200a076828b9fa => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:57.595712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:57.598781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:57.621457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779000c4-4534-4990-85fc-82c44a867dfe admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:57.685522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:57.693097 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:57.694264 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:57.694558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:57.695159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] PATH_INFO: `/v3/projects/9076da63c34c4d24b7651fb9a72d0975` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:57.697121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9076da63c34c4d24b7651fb9a72d0975)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:57.699507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:57.754228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9076da63c34c4d24b7651fb9a72d0975', 'request_id': 'req-710603ff-2501-4ef9-9334-76fb81db41fb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.762701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9076da63c34c4d24b7651fb9a72d0975', 'request_id': 'req-710603ff-2501-4ef9-9334-76fb81db41fb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.769624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9076da63c34c4d24b7651fb9a72d0975', 'request_id': 'req-710603ff-2501-4ef9-9334-76fb81db41fb'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.781149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] The token cache is being invalidate because project 9076da63c34c4d24b7651fb9a72d0975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:34:57.781437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.782116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-710603ff-2501-4ef9-9334-76fb81db41fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:34:57.783753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1182/2347] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:34:57 2022] DELETE /identity/v3/projects/9076da63c34c4d24b7651fb9a72d0975 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:34:57.788821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0907ef-b3c8-4072-a963-a8cd35217fcd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:57.789125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0907ef-b3c8-4072-a963-a8cd35217fcd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:57.789408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0907ef-b3c8-4072-a963-a8cd35217fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:57.809631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-ce0907ef-b3c8-4072-a963-a8cd35217fcd None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:57.810917 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ce0907ef-b3c8-4072-a963-a8cd35217fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:57.881664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1166/2348] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:57.892370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b519df-1140-4fd2-b225-d11c661761e4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:57.892753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b519df-1140-4fd2-b225-d11c661761e4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:57.893176 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b519df-1140-4fd2-b225-d11c661761e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:57.916788 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-92b519df-1140-4fd2-b225-d11c661761e4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:57.918042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-92b519df-1140-4fd2-b225-d11c661761e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:57.927081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1183/2349] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:34:57.932356 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0907ef-b3c8-4072-a963-a8cd35217fcd None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:57.956881 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0907ef-b3c8-4072-a963-a8cd35217fcd None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:57.962078 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef62588-dcfa-4d42-861b-aacfbc14c2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:57.962897 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62588-dcfa-4d42-861b-aacfbc14c2bc admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:57.963188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62588-dcfa-4d42-861b-aacfbc14c2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:57.963530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef62588-dcfa-4d42-861b-aacfbc14c2bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:57.965786 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef62588-dcfa-4d42-861b-aacfbc14c2bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:34:57.966102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef62588-dcfa-4d42-861b-aacfbc14c2bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:57.967811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef62588-dcfa-4d42-861b-aacfbc14c2bc admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:57.984154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1167/2350] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:34:57.988503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b519df-1140-4fd2-b225-d11c661761e4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:58.002404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b519df-1140-4fd2-b225-d11c661761e4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:58.007794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a77142-a312-4f84-90e9-72d241e49725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:58.008508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a77142-a312-4f84-90e9-72d241e49725 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:58.008798 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a77142-a312-4f84-90e9-72d241e49725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:58.009080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a77142-a312-4f84-90e9-72d241e49725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:58.009832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a77142-a312-4f84-90e9-72d241e49725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:58.013278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a77142-a312-4f84-90e9-72d241e49725 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:58.058457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1184/2351] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:34:57 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:58.064272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef62588-dcfa-4d42-861b-aacfbc14c2bc admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:58.084308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef62588-dcfa-4d42-861b-aacfbc14c2bc admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:58.155807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:58.177731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b35d6-da02-4704-b7b6-fb19c321ce1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:58.178911 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b35d6-da02-4704-b7b6-fb19c321ce1e admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:58.179097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b35d6-da02-4704-b7b6-fb19c321ce1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:58.179355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b35d6-da02-4704-b7b6-fb19c321ce1e admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:58.180090 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b35d6-da02-4704-b7b6-fb19c321ce1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:58.182573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b35d6-da02-4704-b7b6-fb19c321ce1e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:58.208777 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1168/2352] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:34:58 2022] POST /identity/v3/users => generated 293 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:34:58.213150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a77142-a312-4f84-90e9-72d241e49725 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:58.231525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a77142-a312-4f84-90e9-72d241e49725 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:58.237617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa863b-391e-4a35-9f71-7a78471dd649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:58.238081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa863b-391e-4a35-9f71-7a78471dd649 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:58.238373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa863b-391e-4a35-9f71-7a78471dd649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:58.238684 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa863b-391e-4a35-9f71-7a78471dd649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:58.239421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa863b-391e-4a35-9f71-7a78471dd649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:58.241156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa863b-391e-4a35-9f71-7a78471dd649 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:58.258760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1185/2353] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:58.274237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b35d6-da02-4704-b7b6-fb19c321ce1e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:58.287189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b35d6-da02-4704-b7b6-fb19c321ce1e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:58.298355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b473e138-3c9d-4bea-8293-0ddd3642a010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:58.298759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473e138-3c9d-4bea-8293-0ddd3642a010 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:58.298942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473e138-3c9d-4bea-8293-0ddd3642a010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:58.299369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473e138-3c9d-4bea-8293-0ddd3642a010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:58.300421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473e138-3c9d-4bea-8293-0ddd3642a010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:58.302832 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b473e138-3c9d-4bea-8293-0ddd3642a010 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:58.326829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1169/2354] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:58.330559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa863b-391e-4a35-9f71-7a78471dd649 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:58.351892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa863b-391e-4a35-9f71-7a78471dd649 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:58.358185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463c00b-99e2-42e9-84bf-5e0f650dda3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:58.359304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463c00b-99e2-42e9-84bf-5e0f650dda3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:58.359607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463c00b-99e2-42e9-84bf-5e0f650dda3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:58.365483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463c00b-99e2-42e9-84bf-5e0f650dda3d admin admin] PATH_INFO: `/v3/projects/da41e8b6cb2b454abb077616783f10c7/users/eab2313be8b7437781de55effdfe932f/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:58.395943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463c00b-99e2-42e9-84bf-5e0f650dda3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=da41e8b6cb2b454abb077616783f10c7, user_id=eab2313be8b7437781de55effdfe932f, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:58.401809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463c00b-99e2-42e9-84bf-5e0f650dda3d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:58.427707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1186/2355] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:58 2022] PUT /identity/v3/projects/da41e8b6cb2b454abb077616783f10c7/users/eab2313be8b7437781de55effdfe932f/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:58.448599 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627eb93-4865-4de0-9ebe-2a5d3b43fa15 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:58.449102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627eb93-4865-4de0-9ebe-2a5d3b43fa15 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:58.449615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627eb93-4865-4de0-9ebe-2a5d3b43fa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:58.496583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3627eb93-4865-4de0-9ebe-2a5d3b43fa15 None None] MFA Rules not processed for user `eab2313be8b7437781de55effdfe932f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:58.498760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3627eb93-4865-4de0-9ebe-2a5d3b43fa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:58.591108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1170/2356] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:58.595727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463c00b-99e2-42e9-84bf-5e0f650dda3d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:58.691489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463c00b-99e2-42e9-84bf-5e0f650dda3d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:58.696813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad75ce-9aa5-45dc-ae27-b0af3e9da925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:58.699113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad75ce-9aa5-45dc-ae27-b0af3e9da925 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:58.699765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad75ce-9aa5-45dc-ae27-b0af3e9da925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:58.699765 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad75ce-9aa5-45dc-ae27-b0af3e9da925 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:58.700848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad75ce-9aa5-45dc-ae27-b0af3e9da925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:58.703091 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad75ce-9aa5-45dc-ae27-b0af3e9da925 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:58.736391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1187/2357] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:34:58 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:58.740672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627eb93-4865-4de0-9ebe-2a5d3b43fa15 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:58.752585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627eb93-4865-4de0-9ebe-2a5d3b43fa15 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:58.824821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff67e2b-bc01-4fed-91fc-aa9415b1458d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:58.825517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff67e2b-bc01-4fed-91fc-aa9415b1458d admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:58.825811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff67e2b-bc01-4fed-91fc-aa9415b1458d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:58.826097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff67e2b-bc01-4fed-91fc-aa9415b1458d admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:58.826858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff67e2b-bc01-4fed-91fc-aa9415b1458d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:58.829052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff67e2b-bc01-4fed-91fc-aa9415b1458d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:58.851806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1171/2358] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:34:58 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:58.856536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad75ce-9aa5-45dc-ae27-b0af3e9da925 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:58.867866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad75ce-9aa5-45dc-ae27-b0af3e9da925 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:58.873679 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d49243-80a0-4d23-8d5a-15ffe88b91ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:58.874276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49243-80a0-4d23-8d5a-15ffe88b91ac admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:58.874564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49243-80a0-4d23-8d5a-15ffe88b91ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:58.874873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d49243-80a0-4d23-8d5a-15ffe88b91ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:58.875622 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d49243-80a0-4d23-8d5a-15ffe88b91ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:58.877268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d49243-80a0-4d23-8d5a-15ffe88b91ac admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:58.879696 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:58.895374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1188/2359] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:58.896513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff67e2b-bc01-4fed-91fc-aa9415b1458d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:58.912411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff67e2b-bc01-4fed-91fc-aa9415b1458d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:58.917621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c4587-c5cb-4fd6-8aa8-241227dd8162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:58.918380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c4587-c5cb-4fd6-8aa8-241227dd8162 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:58.918698 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c4587-c5cb-4fd6-8aa8-241227dd8162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:58.918978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c4587-c5cb-4fd6-8aa8-241227dd8162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:58.919719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c4587-c5cb-4fd6-8aa8-241227dd8162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:58.921333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c4587-c5cb-4fd6-8aa8-241227dd8162 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:58.935059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1172/2360] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:58.939759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d49243-80a0-4d23-8d5a-15ffe88b91ac admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:58.951164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d49243-80a0-4d23-8d5a-15ffe88b91ac admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:58.955930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541880f2-9d6e-4475-8f6a-35970427bfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:58.956837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541880f2-9d6e-4475-8f6a-35970427bfa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:58.957118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541880f2-9d6e-4475-8f6a-35970427bfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:58.957390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541880f2-9d6e-4475-8f6a-35970427bfa6 admin admin] PATH_INFO: `/v3/projects/2a532c8bae744de5b7d822f151979d88/users/b2782a49837b4e64935f714f1622d066/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:58.974252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541880f2-9d6e-4475-8f6a-35970427bfa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a532c8bae744de5b7d822f151979d88, user_id=b2782a49837b4e64935f714f1622d066, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:58.977005 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541880f2-9d6e-4475-8f6a-35970427bfa6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:58.992513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1189/2361] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:58 2022] PUT /identity/v3/projects/2a532c8bae744de5b7d822f151979d88/users/b2782a49837b4e64935f714f1622d066/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:34:59.003694 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2d58d-39f2-4051-9484-59e5db1cfaa3 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:59.003995 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2d58d-39f2-4051-9484-59e5db1cfaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:59.004348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2d58d-39f2-4051-9484-59e5db1cfaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:59.034549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-77c2d58d-39f2-4051-9484-59e5db1cfaa3 None None] MFA Rules not processed for user `b2782a49837b4e64935f714f1622d066`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:59.035906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-77c2d58d-39f2-4051-9484-59e5db1cfaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:59.059095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:59.105938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1173/2362] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:59.110520 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541880f2-9d6e-4475-8f6a-35970427bfa6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:59.176277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541880f2-9d6e-4475-8f6a-35970427bfa6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:59.181753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807ca00-61b9-4b52-935c-0ab94b7023cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:59.183258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807ca00-61b9-4b52-935c-0ab94b7023cf admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:59.183546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807ca00-61b9-4b52-935c-0ab94b7023cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:59.183826 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807ca00-61b9-4b52-935c-0ab94b7023cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:59.184562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807ca00-61b9-4b52-935c-0ab94b7023cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:59.186707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807ca00-61b9-4b52-935c-0ab94b7023cf admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:59.215665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1190/2363] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:34:59 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:59.219849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2d58d-39f2-4051-9484-59e5db1cfaa3 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:59.231524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2d58d-39f2-4051-9484-59e5db1cfaa3 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:59.293358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388e61-ad75-41f3-a15a-26afb90d5c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:59.294042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388e61-ad75-41f3-a15a-26afb90d5c43 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:59.294327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388e61-ad75-41f3-a15a-26afb90d5c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:59.294632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388e61-ad75-41f3-a15a-26afb90d5c43 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:59.295360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6388e61-ad75-41f3-a15a-26afb90d5c43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:59.297306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6388e61-ad75-41f3-a15a-26afb90d5c43 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:59.319059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1174/2364] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:34:59 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:34:59.323129 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807ca00-61b9-4b52-935c-0ab94b7023cf admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:59.333977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807ca00-61b9-4b52-935c-0ab94b7023cf admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:59.338792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95995bd-18d5-47cc-a1b1-3e2aac4a39b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:59.339431 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95995bd-18d5-47cc-a1b1-3e2aac4a39b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:59.339711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95995bd-18d5-47cc-a1b1-3e2aac4a39b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:59.340116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95995bd-18d5-47cc-a1b1-3e2aac4a39b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:59.340719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95995bd-18d5-47cc-a1b1-3e2aac4a39b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:59.342380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95995bd-18d5-47cc-a1b1-3e2aac4a39b5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:59.354721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1191/2365] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:59.359241 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388e61-ad75-41f3-a15a-26afb90d5c43 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:59.369299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388e61-ad75-41f3-a15a-26afb90d5c43 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:59.375168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873414c-82bd-42e7-8df7-890008d23ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:59.376051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873414c-82bd-42e7-8df7-890008d23ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:59.376327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873414c-82bd-42e7-8df7-890008d23ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:59.376598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873414c-82bd-42e7-8df7-890008d23ce4 admin admin] PATH_INFO: `/v3/projects/692f04717635415183ae148f1c631bbc/users/1eb5f9da2c1949f685c7101d422b250f/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:59.393082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873414c-82bd-42e7-8df7-890008d23ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=692f04717635415183ae148f1c631bbc, user_id=1eb5f9da2c1949f685c7101d422b250f, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:59.395803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873414c-82bd-42e7-8df7-890008d23ce4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:59.413083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1175/2366] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:59 2022] PUT /identity/v3/projects/692f04717635415183ae148f1c631bbc/users/1eb5f9da2c1949f685c7101d422b250f/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:34:59.418177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95995bd-18d5-47cc-a1b1-3e2aac4a39b5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:59.428806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95995bd-18d5-47cc-a1b1-3e2aac4a39b5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:59.492490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842baa6f-c976-406e-afa9-1942a4abb955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:59.493936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842baa6f-c976-406e-afa9-1942a4abb955 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:59.494376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842baa6f-c976-406e-afa9-1942a4abb955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:59.495258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842baa6f-c976-406e-afa9-1942a4abb955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:59.496336 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842baa6f-c976-406e-afa9-1942a4abb955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:59.498851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842baa6f-c976-406e-afa9-1942a4abb955 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:59.515484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1192/2367] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:34:59.521906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873414c-82bd-42e7-8df7-890008d23ce4 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:59.542899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873414c-82bd-42e7-8df7-890008d23ce4 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:59.550868 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561579e-568f-4c22-8abe-bebc37f86e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:59.551779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561579e-568f-4c22-8abe-bebc37f86e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:59.552068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561579e-568f-4c22-8abe-bebc37f86e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:59.552357 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561579e-568f-4c22-8abe-bebc37f86e07 admin admin] PATH_INFO: `/v3/projects/692f04717635415183ae148f1c631bbc/users/1eb5f9da2c1949f685c7101d422b250f/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:59.555259 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561579e-568f-4c22-8abe-bebc37f86e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=692f04717635415183ae148f1c631bbc, user_id=1eb5f9da2c1949f685c7101d422b250f, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:59.558915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1561579e-568f-4c22-8abe-bebc37f86e07 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:59.571739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1176/2368] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:34:59 2022] PUT /identity/v3/projects/692f04717635415183ae148f1c631bbc/users/1eb5f9da2c1949f685c7101d422b250f/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:34:59.584873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fae60-4b2a-48cb-a445-95e9aaf0b78a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:59.585165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fae60-4b2a-48cb-a445-95e9aaf0b78a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:59.586095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fae60-4b2a-48cb-a445-95e9aaf0b78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:59.619010 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-924fae60-4b2a-48cb-a445-95e9aaf0b78a None None] MFA Rules not processed for user `1eb5f9da2c1949f685c7101d422b250f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:34:59.620993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-924fae60-4b2a-48cb-a445-95e9aaf0b78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:59.662532 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:34:59.717127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1193/2369] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:34:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:34:59.728460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561579e-568f-4c22-8abe-bebc37f86e07 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:34:59.730776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1561579e-568f-4c22-8abe-bebc37f86e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:34:59.808779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561579e-568f-4c22-8abe-bebc37f86e07 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:34:59.814522 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7618fc-15e9-4497-a709-f899dc48ae0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:34:59.815303 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7618fc-15e9-4497-a709-f899dc48ae0e service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:34:59.815666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7618fc-15e9-4497-a709-f899dc48ae0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:34:59.816018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7618fc-15e9-4497-a709-f899dc48ae0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:34:59.829711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7618fc-15e9-4497-a709-f899dc48ae0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:34:59.831836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7618fc-15e9-4497-a709-f899dc48ae0e service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:34:59.864561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1177/2370] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:34:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:00.457084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fae60-4b2a-48cb-a445-95e9aaf0b78a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:00.458509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-924fae60-4b2a-48cb-a445-95e9aaf0b78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:00.471460 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fae60-4b2a-48cb-a445-95e9aaf0b78a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:00.475257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d59ebe-5f65-40ab-ad23-0aacd7050515 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:00.475913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59ebe-5f65-40ab-ad23-0aacd7050515 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:00.476199 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59ebe-5f65-40ab-ad23-0aacd7050515 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:00.476475 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59ebe-5f65-40ab-ad23-0aacd7050515 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:00.546439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d59ebe-5f65-40ab-ad23-0aacd7050515 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:00.548080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d59ebe-5f65-40ab-ad23-0aacd7050515 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:00.558801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1194/2371] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:06.912110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7618fc-15e9-4497-a709-f899dc48ae0e service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:06.921242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7618fc-15e9-4497-a709-f899dc48ae0e service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:06.925336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be20734-99da-4ca9-a366-6fa414dc37f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:06.925757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be20734-99da-4ca9-a366-6fa414dc37f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:06.926041 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be20734-99da-4ca9-a366-6fa414dc37f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:06.926322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be20734-99da-4ca9-a366-6fa414dc37f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:06.942637 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:06.994699 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be20734-99da-4ca9-a366-6fa414dc37f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:06.996334 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be20734-99da-4ca9-a366-6fa414dc37f3 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:07.010056 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1178/2372] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:09.301009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d59ebe-5f65-40ab-ad23-0aacd7050515 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:09.376533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d59ebe-5f65-40ab-ad23-0aacd7050515 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:09.381580 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:09.382473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:09.382789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:09.383068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] PATH_INFO: `/v3/users/eab2313be8b7437781de55effdfe932f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:09.384286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eab2313be8b7437781de55effdfe932f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:09.386467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:09.454706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eab2313be8b7437781de55effdfe932f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:09.476929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eab2313be8b7437781de55effdfe932f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:09.477702 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eab2313be8b7437781de55effdfe932f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:09.482502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:09.487504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eab2313be8b7437781de55effdfe932f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:09.498606 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eab2313be8b7437781de55effdfe932f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:09.505583 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1195/2373] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:35:09 2022] DELETE /identity/v3/users/eab2313be8b7437781de55effdfe932f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:09.510471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddbfdd-71c6-47f4-b3cd-e515ee044cce None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:09.511123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddbfdd-71c6-47f4-b3cd-e515ee044cce None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:09.511339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddbfdd-71c6-47f4-b3cd-e515ee044cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:09.531426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-38ddbfdd-71c6-47f4-b3cd-e515ee044cce None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:09.532674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-38ddbfdd-71c6-47f4-b3cd-e515ee044cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:09.602568 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1179/2374] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:09.613669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:09.615122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:09.705184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34519228-dba2-4c0d-8da7-5a8a1e0a2f26 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:09.709014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439783ab-f63d-466f-a5f8-f8b6db97865a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:09.709675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439783ab-f63d-466f-a5f8-f8b6db97865a service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:09.709962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439783ab-f63d-466f-a5f8-f8b6db97865a service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:09.710245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439783ab-f63d-466f-a5f8-f8b6db97865a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:09.722155 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439783ab-f63d-466f-a5f8-f8b6db97865a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:09.723688 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439783ab-f63d-466f-a5f8-f8b6db97865a service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:09.740759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1196/2375] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:10.132854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddbfdd-71c6-47f4-b3cd-e515ee044cce None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:10.134388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-38ddbfdd-71c6-47f4-b3cd-e515ee044cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:10.147238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddbfdd-71c6-47f4-b3cd-e515ee044cce None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:10.152634 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:10.153485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:10.153765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:10.154040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] PATH_INFO: `/v3/projects/da41e8b6cb2b454abb077616783f10c7` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:10.155296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da41e8b6cb2b454abb077616783f10c7)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:10.157479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:10.206392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da41e8b6cb2b454abb077616783f10c7', 'request_id': 'req-a5b6ede2-812c-420c-8e6f-8adebb471273'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:10.212110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da41e8b6cb2b454abb077616783f10c7', 'request_id': 'req-a5b6ede2-812c-420c-8e6f-8adebb471273'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:10.219417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da41e8b6cb2b454abb077616783f10c7', 'request_id': 'req-a5b6ede2-812c-420c-8e6f-8adebb471273'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:10.234718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] The token cache is being invalidate because project da41e8b6cb2b454abb077616783f10c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:35:10.235015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:10.236378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:10.237283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1180/2376] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:35:10 2022] DELETE /identity/v3/projects/da41e8b6cb2b454abb077616783f10c7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:10.241993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439783ab-f63d-466f-a5f8-f8b6db97865a service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:10.252099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439783ab-f63d-466f-a5f8-f8b6db97865a service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:10.313107 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:10.313990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:10.314272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:10.314545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] PATH_INFO: `/v3/users/b2782a49837b4e64935f714f1622d066` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:10.315814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2782a49837b4e64935f714f1622d066)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:10.317760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:10.372361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2782a49837b4e64935f714f1622d066'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:10.388810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2782a49837b4e64935f714f1622d066'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:10.389819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2782a49837b4e64935f714f1622d066'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:10.408002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2782a49837b4e64935f714f1622d066'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:10.415838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2782a49837b4e64935f714f1622d066'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:10.422998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1197/2377] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:35:10 2022] DELETE /identity/v3/users/b2782a49837b4e64935f714f1622d066 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:10.944641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:10.946819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:10.962308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6ede2-812c-420c-8e6f-8adebb471273 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:11.031635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:11.062916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:11.063838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:11.064130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:11.064512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] PATH_INFO: `/v3/projects/2a532c8bae744de5b7d822f151979d88` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:11.065695 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a532c8bae744de5b7d822f151979d88)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:11.068014 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:11.125068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a532c8bae744de5b7d822f151979d88', 'request_id': 'req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:11.133209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a532c8bae744de5b7d822f151979d88', 'request_id': 'req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:11.142173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a532c8bae744de5b7d822f151979d88', 'request_id': 'req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:11.157957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] The token cache is being invalidate because project 2a532c8bae744de5b7d822f151979d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:35:11.158254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:11.159228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:11.160579 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1181/2378] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:35:10 2022] DELETE /identity/v3/projects/2a532c8bae744de5b7d822f151979d88 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:11.164596 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:11.166472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:11.182523 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc7d01-b172-4345-af27-75014739c6b3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:11.189637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:11.270812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38981ca1-31d7-4461-9a4c-5cc240d9927e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:11.272452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38981ca1-31d7-4461-9a4c-5cc240d9927e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:11.273096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38981ca1-31d7-4461-9a4c-5cc240d9927e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:11.273616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38981ca1-31d7-4461-9a4c-5cc240d9927e admin admin] PATH_INFO: `/v3/users/1eb5f9da2c1949f685c7101d422b250f` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:11.276161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38981ca1-31d7-4461-9a4c-5cc240d9927e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb5f9da2c1949f685c7101d422b250f)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:11.280182 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38981ca1-31d7-4461-9a4c-5cc240d9927e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:11.362809 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-38981ca1-31d7-4461-9a4c-5cc240d9927e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb5f9da2c1949f685c7101d422b250f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:11.378227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-38981ca1-31d7-4461-9a4c-5cc240d9927e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb5f9da2c1949f685c7101d422b250f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:11.379304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-38981ca1-31d7-4461-9a4c-5cc240d9927e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb5f9da2c1949f685c7101d422b250f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:11.389569 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-38981ca1-31d7-4461-9a4c-5cc240d9927e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb5f9da2c1949f685c7101d422b250f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:11.400115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-38981ca1-31d7-4461-9a4c-5cc240d9927e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb5f9da2c1949f685c7101d422b250f'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:11.415835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1198/2379] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:35:11 2022] DELETE /identity/v3/users/1eb5f9da2c1949f685c7101d422b250f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:11.829264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:11.830891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:11.855423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43f22b-c2e4-4d2b-a6ff-f1b931742b0e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:11.932464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:11.934051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:11.934399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:11.934673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] PATH_INFO: `/v3/projects/692f04717635415183ae148f1c631bbc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:11.936705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=692f04717635415183ae148f1c631bbc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:11.938413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:11.994296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '692f04717635415183ae148f1c631bbc', 'request_id': 'req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:12.002657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '692f04717635415183ae148f1c631bbc', 'request_id': 'req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:12.009635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '692f04717635415183ae148f1c631bbc', 'request_id': 'req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:12.023088 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:12.030275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] The token cache is being invalidate because project 692f04717635415183ae148f1c631bbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:35:12.030851 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:12.032553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7cc62ce3-94b8-4c6f-8fcf-2dd2aa20f4f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:12.034647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1182/2380] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:35:11 2022] DELETE /identity/v3/projects/692f04717635415183ae148f1c631bbc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:12.041499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb349e-b359-44aa-9ede-0bfb42a3c444 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:12.043050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb349e-b359-44aa-9ede-0bfb42a3c444 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:12.043540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb349e-b359-44aa-9ede-0bfb42a3c444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:12.066929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3edb349e-b359-44aa-9ede-0bfb42a3c444 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:12.068325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3edb349e-b359-44aa-9ede-0bfb42a3c444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:12.165651 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1199/2381] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:35:12.232693 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61399c72-27f2-465b-91ea-657bf8baa05f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:12.232980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61399c72-27f2-465b-91ea-657bf8baa05f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:12.233261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61399c72-27f2-465b-91ea-657bf8baa05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:12.255882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-61399c72-27f2-465b-91ea-657bf8baa05f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:12.257479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-61399c72-27f2-465b-91ea-657bf8baa05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:12.266394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1183/2382] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:12.270691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb349e-b359-44aa-9ede-0bfb42a3c444 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:12.281349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb349e-b359-44aa-9ede-0bfb42a3c444 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:12.287104 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628db5b9-0a27-4744-af3e-66d2179ae4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:12.287990 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628db5b9-0a27-4744-af3e-66d2179ae4df admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:12.288277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628db5b9-0a27-4744-af3e-66d2179ae4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:12.288558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628db5b9-0a27-4744-af3e-66d2179ae4df admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:12.289307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628db5b9-0a27-4744-af3e-66d2179ae4df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:35:12.289619 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628db5b9-0a27-4744-af3e-66d2179ae4df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:12.291353 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628db5b9-0a27-4744-af3e-66d2179ae4df admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:12.305152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1200/2383] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:35:12.310557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61399c72-27f2-465b-91ea-657bf8baa05f None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:12.321540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61399c72-27f2-465b-91ea-657bf8baa05f None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:12.326982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129f5f1-3237-48d1-83da-92350437173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:12.327661 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129f5f1-3237-48d1-83da-92350437173a admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:12.327948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129f5f1-3237-48d1-83da-92350437173a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:12.328223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129f5f1-3237-48d1-83da-92350437173a admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:12.329059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129f5f1-3237-48d1-83da-92350437173a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:12.331108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129f5f1-3237-48d1-83da-92350437173a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:12.365262 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1184/2384] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:35:12 2022] POST /identity/v3/projects => generated 399 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:35:12.366556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628db5b9-0a27-4744-af3e-66d2179ae4df admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:12.378916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628db5b9-0a27-4744-af3e-66d2179ae4df admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:12.393644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:12.447987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb34a5-74f7-457c-938a-1e15d6ed7fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:12.448872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb34a5-74f7-457c-938a-1e15d6ed7fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:12.449168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb34a5-74f7-457c-938a-1e15d6ed7fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:12.449479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb34a5-74f7-457c-938a-1e15d6ed7fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:12.450188 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb34a5-74f7-457c-938a-1e15d6ed7fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:12.452224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb34a5-74f7-457c-938a-1e15d6ed7fd6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:12.473707 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1201/2385] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:35:12 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:35:12.478121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129f5f1-3237-48d1-83da-92350437173a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:12.490922 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129f5f1-3237-48d1-83da-92350437173a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:12.496145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46849b7f-0a33-42cf-8efc-93b3d4ee30c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:12.496867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46849b7f-0a33-42cf-8efc-93b3d4ee30c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:12.497134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46849b7f-0a33-42cf-8efc-93b3d4ee30c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:12.497426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46849b7f-0a33-42cf-8efc-93b3d4ee30c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:12.498154 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46849b7f-0a33-42cf-8efc-93b3d4ee30c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:12.499864 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46849b7f-0a33-42cf-8efc-93b3d4ee30c9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:12.511277 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1185/2386] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:35:12.516206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb34a5-74f7-457c-938a-1e15d6ed7fd6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:12.530876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb34a5-74f7-457c-938a-1e15d6ed7fd6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:12.536898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312ba3f-83fd-4185-bdde-f0549d5d0720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:12.537577 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312ba3f-83fd-4185-bdde-f0549d5d0720 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:12.537864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312ba3f-83fd-4185-bdde-f0549d5d0720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:12.538143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312ba3f-83fd-4185-bdde-f0549d5d0720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:12.538942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312ba3f-83fd-4185-bdde-f0549d5d0720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:12.540593 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312ba3f-83fd-4185-bdde-f0549d5d0720 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:12.557962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1202/2387] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:35:12.563103 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46849b7f-0a33-42cf-8efc-93b3d4ee30c9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:12.573573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46849b7f-0a33-42cf-8efc-93b3d4ee30c9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:12.578519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8e5c-ca0f-4f54-b827-1137f9782e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:12.579511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8e5c-ca0f-4f54-b827-1137f9782e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:12.579971 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8e5c-ca0f-4f54-b827-1137f9782e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:12.580260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8e5c-ca0f-4f54-b827-1137f9782e95 admin admin] PATH_INFO: `/v3/projects/e95adb7182f940e7bb4ad16ce754dfeb/users/a6323bb23d464c60b55641db8c498341/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:12.597821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f8e5c-ca0f-4f54-b827-1137f9782e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e95adb7182f940e7bb4ad16ce754dfeb, user_id=a6323bb23d464c60b55641db8c498341, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:12.601299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f8e5c-ca0f-4f54-b827-1137f9782e95 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:12.633089 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1186/2388] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:35:12 2022] PUT /identity/v3/projects/e95adb7182f940e7bb4ad16ce754dfeb/users/a6323bb23d464c60b55641db8c498341/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:12.637147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2a84-110e-4ea2-bf9f-2a6b675c137b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:12.637618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2a84-110e-4ea2-bf9f-2a6b675c137b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:12.637968 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e2a84-110e-4ea2-bf9f-2a6b675c137b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:12.674721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-fc5e2a84-110e-4ea2-bf9f-2a6b675c137b None None] MFA Rules not processed for user `a6323bb23d464c60b55641db8c498341`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:12.677271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fc5e2a84-110e-4ea2-bf9f-2a6b675c137b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:12.773577 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1203/2389] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:12.779183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8e5c-ca0f-4f54-b827-1137f9782e95 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:12.858358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8e5c-ca0f-4f54-b827-1137f9782e95 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:12.863310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447add2-2e46-48a6-8834-dc57c791deb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:12.864056 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447add2-2e46-48a6-8834-dc57c791deb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:12.864419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447add2-2e46-48a6-8834-dc57c791deb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:12.864773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447add2-2e46-48a6-8834-dc57c791deb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:12.865544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447add2-2e46-48a6-8834-dc57c791deb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:12.867583 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c447add2-2e46-48a6-8834-dc57c791deb3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:12.909675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1187/2390] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:35:12 2022] POST /identity/v3/projects => generated 399 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:35:12.910435 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e2a84-110e-4ea2-bf9f-2a6b675c137b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:12.924213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e2a84-110e-4ea2-bf9f-2a6b675c137b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:12.990101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c791a5b-f2a2-46e2-8669-abc7d17a783e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:12.990813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c791a5b-f2a2-46e2-8669-abc7d17a783e admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:12.991101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c791a5b-f2a2-46e2-8669-abc7d17a783e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:12.991407 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c791a5b-f2a2-46e2-8669-abc7d17a783e admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:12.992103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c791a5b-f2a2-46e2-8669-abc7d17a783e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:12.994109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c791a5b-f2a2-46e2-8669-abc7d17a783e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:13.028535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1204/2391] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:35:12 2022] POST /identity/v3/users => generated 306 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:35:13.032541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447add2-2e46-48a6-8834-dc57c791deb3 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:13.044220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447add2-2e46-48a6-8834-dc57c791deb3 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:13.049261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71e9ca-117a-4a05-be40-91abcc1f3a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:13.050022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71e9ca-117a-4a05-be40-91abcc1f3a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:13.050377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71e9ca-117a-4a05-be40-91abcc1f3a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:13.050745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71e9ca-117a-4a05-be40-91abcc1f3a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:13.051491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71e9ca-117a-4a05-be40-91abcc1f3a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:13.053278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71e9ca-117a-4a05-be40-91abcc1f3a71 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:13.065374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1188/2392] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:35:13.069456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c791a5b-f2a2-46e2-8669-abc7d17a783e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:13.083643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c791a5b-f2a2-46e2-8669-abc7d17a783e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:13.088538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e49f89-2dad-43ca-8734-810c162c2a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:13.090143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e49f89-2dad-43ca-8734-810c162c2a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:13.090493 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e49f89-2dad-43ca-8734-810c162c2a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:13.090868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e49f89-2dad-43ca-8734-810c162c2a87 admin admin] PATH_INFO: `/v3/projects/46fec70112134c68af6e954d6fe236a9/users/500e2b95f8f04dc1ab294a425f81b5f3/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:13.103680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:13.110179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e49f89-2dad-43ca-8734-810c162c2a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46fec70112134c68af6e954d6fe236a9, user_id=500e2b95f8f04dc1ab294a425f81b5f3, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:13.113240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e49f89-2dad-43ca-8734-810c162c2a87 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:13.131380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079c1bc-397d-475b-b135-22a5c5c0812b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:13.131648 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079c1bc-397d-475b-b135-22a5c5c0812b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:13.131982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079c1bc-397d-475b-b135-22a5c5c0812b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:13.135887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1205/2393] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:35:13 2022] PUT /identity/v3/projects/46fec70112134c68af6e954d6fe236a9/users/500e2b95f8f04dc1ab294a425f81b5f3/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:13.140361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e49f89-2dad-43ca-8734-810c162c2a87 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:13.163070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-9079c1bc-397d-475b-b135-22a5c5c0812b None None] MFA Rules not processed for user `b2326ff15f294e15806d8a564770b35e`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:13.164301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9079c1bc-397d-475b-b135-22a5c5c0812b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:13.209224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e49f89-2dad-43ca-8734-810c162c2a87 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:13.214255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c835d2-72fb-4d61-82fd-9307dbd2463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:13.214938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c835d2-72fb-4d61-82fd-9307dbd2463f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:13.215328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c835d2-72fb-4d61-82fd-9307dbd2463f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:13.215498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c835d2-72fb-4d61-82fd-9307dbd2463f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:13.216221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c835d2-72fb-4d61-82fd-9307dbd2463f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:13.217832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c835d2-72fb-4d61-82fd-9307dbd2463f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:13.223746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1189/2394] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:13.231058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1206/2395] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:35:13.232152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079c1bc-397d-475b-b135-22a5c5c0812b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:13.243250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079c1bc-397d-475b-b135-22a5c5c0812b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:13.248606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4bf90-8a7c-4598-a1be-099f8e9c5d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:13.249502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4bf90-8a7c-4598-a1be-099f8e9c5d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:13.249785 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4bf90-8a7c-4598-a1be-099f8e9c5d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:13.250064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4bf90-8a7c-4598-a1be-099f8e9c5d7d admin admin] PATH_INFO: `/v3/projects/46fec70112134c68af6e954d6fe236a9/users/500e2b95f8f04dc1ab294a425f81b5f3/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:13.252434 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4bf90-8a7c-4598-a1be-099f8e9c5d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=46fec70112134c68af6e954d6fe236a9, user_id=500e2b95f8f04dc1ab294a425f81b5f3, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:13.257245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4bf90-8a7c-4598-a1be-099f8e9c5d7d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:13.269514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1190/2396] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:35:13 2022] PUT /identity/v3/projects/46fec70112134c68af6e954d6fe236a9/users/500e2b95f8f04dc1ab294a425f81b5f3/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:13.280482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06580d44-66ff-4d93-bd5d-317abd14d3e2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:13.280778 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06580d44-66ff-4d93-bd5d-317abd14d3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:13.281064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06580d44-66ff-4d93-bd5d-317abd14d3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:13.309052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-06580d44-66ff-4d93-bd5d-317abd14d3e2 None None] MFA Rules not processed for user `500e2b95f8f04dc1ab294a425f81b5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:13.310302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-06580d44-66ff-4d93-bd5d-317abd14d3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:13.382272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1207/2397] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:13 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:35:13.397455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4bf90-8a7c-4598-a1be-099f8e9c5d7d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:13.399215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-73a4bf90-8a7c-4598-a1be-099f8e9c5d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:13.405027 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:13.462297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4bf90-8a7c-4598-a1be-099f8e9c5d7d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:13.466272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6d946-56a7-4c16-b771-224149149fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:13.467022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6d946-56a7-4c16-b771-224149149fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:13.467375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6d946-56a7-4c16-b771-224149149fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:13.467718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f6d946-56a7-4c16-b771-224149149fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:13.477390 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6d946-56a7-4c16-b771-224149149fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:13.478964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f6d946-56a7-4c16-b771-224149149fe3 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:13.492466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1191/2398] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:13 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:13.570090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06580d44-66ff-4d93-bd5d-317abd14d3e2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:13.581234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06580d44-66ff-4d93-bd5d-317abd14d3e2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:13.584902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb192ef2-8a6a-4252-9e13-c4ef26d3500e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:13.585721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb192ef2-8a6a-4252-9e13-c4ef26d3500e service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:13.586035 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb192ef2-8a6a-4252-9e13-c4ef26d3500e service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:13.586367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb192ef2-8a6a-4252-9e13-c4ef26d3500e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:13.649908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb192ef2-8a6a-4252-9e13-c4ef26d3500e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:13.651546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb192ef2-8a6a-4252-9e13-c4ef26d3500e service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:13.661368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1208/2399] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:19.156557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6d946-56a7-4c16-b771-224149149fe3 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:19.228021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f6d946-56a7-4c16-b771-224149149fe3 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:19.233396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:19.234290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:19.234617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:19.234890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] PATH_INFO: `/v3/users/a6323bb23d464c60b55641db8c498341` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:19.236145 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6323bb23d464c60b55641db8c498341)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:19.238114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:19.304953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6323bb23d464c60b55641db8c498341'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:19.313845 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6323bb23d464c60b55641db8c498341'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:19.325778 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6323bb23d464c60b55641db8c498341'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:19.343014 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6323bb23d464c60b55641db8c498341'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:19.343876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6323bb23d464c60b55641db8c498341'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:19.350214 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1192/2400] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:35:19 2022] DELETE /identity/v3/users/a6323bb23d464c60b55641db8c498341 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:19.355685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200715-f5ea-4299-8c9d-e7a9cf7cf8ad None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:19.356252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200715-f5ea-4299-8c9d-e7a9cf7cf8ad None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:19.356252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2200715-f5ea-4299-8c9d-e7a9cf7cf8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:19.374724 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f2200715-f5ea-4299-8c9d-e7a9cf7cf8ad None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:19.375977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f2200715-f5ea-4299-8c9d-e7a9cf7cf8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:19.380492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:19.466019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1209/2401] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:19.480683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:19.482633 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:19.518142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:19.559002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d157a-bf79-473a-8b57-f8e91f16e5d3 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:19.562820 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f797-513a-4d34-8f35-be15e5ceec92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:19.563505 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f797-513a-4d34-8f35-be15e5ceec92 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:19.563794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f797-513a-4d34-8f35-be15e5ceec92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:19.564077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f797-513a-4d34-8f35-be15e5ceec92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:19.575144 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7f797-513a-4d34-8f35-be15e5ceec92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:19.577083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7f797-513a-4d34-8f35-be15e5ceec92 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:19.588020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1193/2402] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:19.906505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2200715-f5ea-4299-8c9d-e7a9cf7cf8ad None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:19.908392 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f2200715-f5ea-4299-8c9d-e7a9cf7cf8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:19.922413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2200715-f5ea-4299-8c9d-e7a9cf7cf8ad None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:19.946807 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:19.947640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:19.948090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:19.948270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] PATH_INFO: `/v3/projects/e95adb7182f940e7bb4ad16ce754dfeb` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:19.949751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e95adb7182f940e7bb4ad16ce754dfeb)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:19.953759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:20.021785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e95adb7182f940e7bb4ad16ce754dfeb', 'request_id': 'req-a0c80b35-ad0e-42b7-90df-18c86fc390d1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.032747 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e95adb7182f940e7bb4ad16ce754dfeb', 'request_id': 'req-a0c80b35-ad0e-42b7-90df-18c86fc390d1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.040654 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e95adb7182f940e7bb4ad16ce754dfeb', 'request_id': 'req-a0c80b35-ad0e-42b7-90df-18c86fc390d1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.060372 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] The token cache is being invalidate because project e95adb7182f940e7bb4ad16ce754dfeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:35:20.060923 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.062041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.064810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1210/2403] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:35:19 2022] DELETE /identity/v3/projects/e95adb7182f940e7bb4ad16ce754dfeb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:20.069188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f797-513a-4d34-8f35-be15e5ceec92 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:20.082311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f797-513a-4d34-8f35-be15e5ceec92 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:20.155523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44abe87-9a5b-4c79-b459-314fa4d24bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:20.156417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44abe87-9a5b-4c79-b459-314fa4d24bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:20.156709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44abe87-9a5b-4c79-b459-314fa4d24bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:20.157331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44abe87-9a5b-4c79-b459-314fa4d24bc6 admin admin] PATH_INFO: `/v3/users/500e2b95f8f04dc1ab294a425f81b5f3` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:20.158250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44abe87-9a5b-4c79-b459-314fa4d24bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=500e2b95f8f04dc1ab294a425f81b5f3)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:20.160251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44abe87-9a5b-4c79-b459-314fa4d24bc6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:20.212528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f44abe87-9a5b-4c79-b459-314fa4d24bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500e2b95f8f04dc1ab294a425f81b5f3'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.220757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f44abe87-9a5b-4c79-b459-314fa4d24bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500e2b95f8f04dc1ab294a425f81b5f3'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.228915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f44abe87-9a5b-4c79-b459-314fa4d24bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500e2b95f8f04dc1ab294a425f81b5f3'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.244516 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f44abe87-9a5b-4c79-b459-314fa4d24bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500e2b95f8f04dc1ab294a425f81b5f3'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.245870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f44abe87-9a5b-4c79-b459-314fa4d24bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500e2b95f8f04dc1ab294a425f81b5f3'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.255368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1194/2404] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:35:20 2022] DELETE /identity/v3/users/500e2b95f8f04dc1ab294a425f81b5f3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:20.598556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:20.600069 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:20.613644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c80b35-ad0e-42b7-90df-18c86fc390d1 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:20.676834 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:20.677705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:20.677994 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:20.678316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] PATH_INFO: `/v3/projects/46fec70112134c68af6e954d6fe236a9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:20.685684 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46fec70112134c68af6e954d6fe236a9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:20.687711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:20.734565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:20.749422 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46fec70112134c68af6e954d6fe236a9', 'request_id': 'req-43aaf11c-5307-4869-8fae-a5608aa90261'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.757850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46fec70112134c68af6e954d6fe236a9', 'request_id': 'req-43aaf11c-5307-4869-8fae-a5608aa90261'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.766369 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46fec70112134c68af6e954d6fe236a9', 'request_id': 'req-43aaf11c-5307-4869-8fae-a5608aa90261'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.778310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] The token cache is being invalidate because project 46fec70112134c68af6e954d6fe236a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:35:20.778548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.779210 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-43aaf11c-5307-4869-8fae-a5608aa90261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:20.780763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1211/2405] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:35:20 2022] DELETE /identity/v3/projects/46fec70112134c68af6e954d6fe236a9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:20.785990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65a548-4cb0-4c28-b160-6c013e38ca61 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:20.786285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65a548-4cb0-4c28-b160-6c013e38ca61 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:20.786566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65a548-4cb0-4c28-b160-6c013e38ca61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:20.810731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-fa65a548-4cb0-4c28-b160-6c013e38ca61 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:20.812024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fa65a548-4cb0-4c28-b160-6c013e38ca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:20.857399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:20.876763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1195/2406] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:20.884950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973aa12-e410-456f-8454-6860d35f0868 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:20.885244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973aa12-e410-456f-8454-6860d35f0868 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:20.885554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973aa12-e410-456f-8454-6860d35f0868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:20.905965 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0973aa12-e410-456f-8454-6860d35f0868 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:20.907550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0973aa12-e410-456f-8454-6860d35f0868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:20.916136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1212/2407] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:20.921268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65a548-4cb0-4c28-b160-6c013e38ca61 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:20.931327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65a548-4cb0-4c28-b160-6c013e38ca61 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:20.936108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf3292-f894-4213-af92-405908d29426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:20.936791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf3292-f894-4213-af92-405908d29426 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:20.937079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf3292-f894-4213-af92-405908d29426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:20.937358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf3292-f894-4213-af92-405908d29426 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:20.938261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf3292-f894-4213-af92-405908d29426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:35:20.938611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf3292-f894-4213-af92-405908d29426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:20.940305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf3292-f894-4213-af92-405908d29426 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:20.953648 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1196/2408] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:35:20.958358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973aa12-e410-456f-8454-6860d35f0868 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:20.969144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973aa12-e410-456f-8454-6860d35f0868 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:20.974144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65532bdc-46b3-4aab-8017-71d3b890d53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:20.974850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65532bdc-46b3-4aab-8017-71d3b890d53e admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:20.975173 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65532bdc-46b3-4aab-8017-71d3b890d53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:20.975455 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65532bdc-46b3-4aab-8017-71d3b890d53e admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:20.976213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65532bdc-46b3-4aab-8017-71d3b890d53e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:20.978177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65532bdc-46b3-4aab-8017-71d3b890d53e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.009916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1213/2409] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:35:20 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:35:21.012223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf3292-f894-4213-af92-405908d29426 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:21.023788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf3292-f894-4213-af92-405908d29426 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:21.084577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb9101-f420-4559-9fdd-ac3f195729fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:21.085261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb9101-f420-4559-9fdd-ac3f195729fa admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.085548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb9101-f420-4559-9fdd-ac3f195729fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.085838 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb9101-f420-4559-9fdd-ac3f195729fa admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.086722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb9101-f420-4559-9fdd-ac3f195729fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:21.089276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb9101-f420-4559-9fdd-ac3f195729fa admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.110936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1197/2410] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:35:21 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:35:21.115265 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65532bdc-46b3-4aab-8017-71d3b890d53e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:21.125832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65532bdc-46b3-4aab-8017-71d3b890d53e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:21.130564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7aa11-443e-44e5-ba62-2d605b7b6c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:21.131259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7aa11-443e-44e5-ba62-2d605b7b6c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.131555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7aa11-443e-44e5-ba62-2d605b7b6c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.131840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7aa11-443e-44e5-ba62-2d605b7b6c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.132603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7aa11-443e-44e5-ba62-2d605b7b6c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:21.134258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7aa11-443e-44e5-ba62-2d605b7b6c6d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.147674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1214/2411] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:35:21.152754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb9101-f420-4559-9fdd-ac3f195729fa admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:21.163273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb9101-f420-4559-9fdd-ac3f195729fa admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:21.168232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5564311-9e7f-47ea-915e-1d9a280cc128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:21.168900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5564311-9e7f-47ea-915e-1d9a280cc128 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.169191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5564311-9e7f-47ea-915e-1d9a280cc128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.169473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5564311-9e7f-47ea-915e-1d9a280cc128 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.170208 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5564311-9e7f-47ea-915e-1d9a280cc128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:21.171887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5564311-9e7f-47ea-915e-1d9a280cc128 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.182452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1198/2412] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:35:21.186837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7aa11-443e-44e5-ba62-2d605b7b6c6d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:21.196813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7aa11-443e-44e5-ba62-2d605b7b6c6d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:21.202507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de756ba5-6e0f-4523-ad43-a27d921a7dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:21.203455 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756ba5-6e0f-4523-ad43-a27d921a7dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.203738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756ba5-6e0f-4523-ad43-a27d921a7dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.204011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de756ba5-6e0f-4523-ad43-a27d921a7dde admin admin] PATH_INFO: `/v3/projects/c78029c7de4144d58520e38c17a47dab/users/0b08a32abfb54ed6aada27ae5cdae0ee/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.220738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de756ba5-6e0f-4523-ad43-a27d921a7dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78029c7de4144d58520e38c17a47dab, user_id=0b08a32abfb54ed6aada27ae5cdae0ee, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:21.223687 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de756ba5-6e0f-4523-ad43-a27d921a7dde admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.237419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1215/2413] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:35:21 2022] PUT /identity/v3/projects/c78029c7de4144d58520e38c17a47dab/users/0b08a32abfb54ed6aada27ae5cdae0ee/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:21.248096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ddfe5-b76a-4be7-a7df-248a5750a71d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.248472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ddfe5-b76a-4be7-a7df-248a5750a71d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.248767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ddfe5-b76a-4be7-a7df-248a5750a71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.279139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7e7ddfe5-b76a-4be7-a7df-248a5750a71d None None] MFA Rules not processed for user `0b08a32abfb54ed6aada27ae5cdae0ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:21.280445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7e7ddfe5-b76a-4be7-a7df-248a5750a71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:21.353854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1199/2414] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:35:21.358546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de756ba5-6e0f-4523-ad43-a27d921a7dde admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:21.422508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de756ba5-6e0f-4523-ad43-a27d921a7dde admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:21.427397 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c75df7-732c-48ff-b346-c018bca8630a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:21.428141 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75df7-732c-48ff-b346-c018bca8630a admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.428376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75df7-732c-48ff-b346-c018bca8630a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.428744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75df7-732c-48ff-b346-c018bca8630a admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.429357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c75df7-732c-48ff-b346-c018bca8630a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:21.431355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c75df7-732c-48ff-b346-c018bca8630a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.465833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1216/2415] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:35:21 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:35:21.466990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ddfe5-b76a-4be7-a7df-248a5750a71d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:21.477725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ddfe5-b76a-4be7-a7df-248a5750a71d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:21.521726 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:21.540017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97b678-7e74-40ed-a2e7-55e10a11ed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:21.540710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97b678-7e74-40ed-a2e7-55e10a11ed6a admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.541001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97b678-7e74-40ed-a2e7-55e10a11ed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.541282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97b678-7e74-40ed-a2e7-55e10a11ed6a admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.543119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97b678-7e74-40ed-a2e7-55e10a11ed6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:21.545073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97b678-7e74-40ed-a2e7-55e10a11ed6a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.566310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1200/2416] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:35:21 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:35:21.570313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c75df7-732c-48ff-b346-c018bca8630a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:21.581617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c75df7-732c-48ff-b346-c018bca8630a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:21.586388 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d599b-bfcc-446a-8041-cf5ff602586f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:21.587117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d599b-bfcc-446a-8041-cf5ff602586f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.587437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d599b-bfcc-446a-8041-cf5ff602586f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.587720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d599b-bfcc-446a-8041-cf5ff602586f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.589014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d599b-bfcc-446a-8041-cf5ff602586f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:21.590939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d599b-bfcc-446a-8041-cf5ff602586f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.602106 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1217/2417] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:35:21.607120 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97b678-7e74-40ed-a2e7-55e10a11ed6a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:21.617500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97b678-7e74-40ed-a2e7-55e10a11ed6a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:21.622389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b2fcb-1e37-430d-83fd-263072243444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:21.623319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b2fcb-1e37-430d-83fd-263072243444 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.623605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b2fcb-1e37-430d-83fd-263072243444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.623906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647b2fcb-1e37-430d-83fd-263072243444 admin admin] PATH_INFO: `/v3/projects/b3f81357301b4692a7bbc4fee7e44398/users/d162b97496c24118b283e0be3f8d2dd8/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.640081 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b2fcb-1e37-430d-83fd-263072243444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f81357301b4692a7bbc4fee7e44398, user_id=d162b97496c24118b283e0be3f8d2dd8, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:21.642867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647b2fcb-1e37-430d-83fd-263072243444 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.657040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1201/2418] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:35:21 2022] PUT /identity/v3/projects/b3f81357301b4692a7bbc4fee7e44398/users/d162b97496c24118b283e0be3f8d2dd8/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:35:21.661148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d599b-bfcc-446a-8041-cf5ff602586f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:21.671995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d599b-bfcc-446a-8041-cf5ff602586f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:21.702099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:21.731150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b714916-e187-47d6-9210-d6a6e8573a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:21.731823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b714916-e187-47d6-9210-d6a6e8573a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.732103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b714916-e187-47d6-9210-d6a6e8573a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.732387 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b714916-e187-47d6-9210-d6a6e8573a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.733101 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b714916-e187-47d6-9210-d6a6e8573a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:21.734740 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b714916-e187-47d6-9210-d6a6e8573a23 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.744649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1218/2419] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:35:21.750628 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b2fcb-1e37-430d-83fd-263072243444 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:21.762420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647b2fcb-1e37-430d-83fd-263072243444 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:21.767280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d8c78-b862-4737-af73-1a95270f2ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:21.768155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d8c78-b862-4737-af73-1a95270f2ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.768439 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d8c78-b862-4737-af73-1a95270f2ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.768712 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d8c78-b862-4737-af73-1a95270f2ecb admin admin] PATH_INFO: `/v3/projects/b3f81357301b4692a7bbc4fee7e44398/users/d162b97496c24118b283e0be3f8d2dd8/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.770683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d8c78-b862-4737-af73-1a95270f2ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f81357301b4692a7bbc4fee7e44398, user_id=d162b97496c24118b283e0be3f8d2dd8, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:21.773414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d8c78-b862-4737-af73-1a95270f2ecb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.784479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1202/2420] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:35:21 2022] PUT /identity/v3/projects/b3f81357301b4692a7bbc4fee7e44398/users/d162b97496c24118b283e0be3f8d2dd8/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:21.795881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8a5ed-b6a1-41f4-a4c5-681bea0be0ca None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.796158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8a5ed-b6a1-41f4-a4c5-681bea0be0ca None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.796440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8a5ed-b6a1-41f4-a4c5-681bea0be0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.822300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3be8a5ed-b6a1-41f4-a4c5-681bea0be0ca None None] MFA Rules not processed for user `d162b97496c24118b283e0be3f8d2dd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:21.824170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3be8a5ed-b6a1-41f4-a4c5-681bea0be0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:21.892793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1219/2421] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:21.904476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d8c78-b862-4737-af73-1a95270f2ecb admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:21.906215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-142d8c78-b862-4737-af73-1a95270f2ecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:21.965590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d8c78-b862-4737-af73-1a95270f2ecb admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:21.969261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b45fa-4e1b-49ed-a053-592bf1cbff43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:21.969905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b45fa-4e1b-49ed-a053-592bf1cbff43 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:21.970192 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b45fa-4e1b-49ed-a053-592bf1cbff43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:21.970472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b45fa-4e1b-49ed-a053-592bf1cbff43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:21.979649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b45fa-4e1b-49ed-a053-592bf1cbff43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:21.981129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b45fa-4e1b-49ed-a053-592bf1cbff43 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:21.990693 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1203/2422] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:22.412259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8a5ed-b6a1-41f4-a4c5-681bea0be0ca None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:22.413738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3be8a5ed-b6a1-41f4-a4c5-681bea0be0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:22.421854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b45fa-4e1b-49ed-a053-592bf1cbff43 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:22.486764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b45fa-4e1b-49ed-a053-592bf1cbff43 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:22.491338 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817020df-4baf-46b1-878f-bc2974545a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:22.492233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817020df-4baf-46b1-878f-bc2974545a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:22.492513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817020df-4baf-46b1-878f-bc2974545a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:22.492788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817020df-4baf-46b1-878f-bc2974545a0e admin admin] PATH_INFO: `/v3/users/0b08a32abfb54ed6aada27ae5cdae0ee` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:22.493996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817020df-4baf-46b1-878f-bc2974545a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b08a32abfb54ed6aada27ae5cdae0ee)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:22.496730 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817020df-4baf-46b1-878f-bc2974545a0e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:22.523105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8a5ed-b6a1-41f4-a4c5-681bea0be0ca None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:22.529536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607ef90-2b23-4710-af72-03fe06e1034f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:22.530283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607ef90-2b23-4710-af72-03fe06e1034f service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:22.530646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607ef90-2b23-4710-af72-03fe06e1034f service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:22.530915 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607ef90-2b23-4710-af72-03fe06e1034f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:22.534400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d607ef90-2b23-4710-af72-03fe06e1034f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:22.540790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:22.548296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-817020df-4baf-46b1-878f-bc2974545a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b08a32abfb54ed6aada27ae5cdae0ee'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:22.556791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-817020df-4baf-46b1-878f-bc2974545a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b08a32abfb54ed6aada27ae5cdae0ee'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:22.564941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:22.565660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-817020df-4baf-46b1-878f-bc2974545a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b08a32abfb54ed6aada27ae5cdae0ee'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:22.578946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-817020df-4baf-46b1-878f-bc2974545a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b08a32abfb54ed6aada27ae5cdae0ee'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:22.579662 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-817020df-4baf-46b1-878f-bc2974545a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b08a32abfb54ed6aada27ae5cdae0ee'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:22.585612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1204/2423] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:35:22 2022] DELETE /identity/v3/users/0b08a32abfb54ed6aada27ae5cdae0ee => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:35:22.589999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8297-59f4-4c7d-b4ac-7f69eb22123d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:22.590291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8297-59f4-4c7d-b4ac-7f69eb22123d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:22.590597 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c8297-59f4-4c7d-b4ac-7f69eb22123d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:22.601113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607ef90-2b23-4710-af72-03fe06e1034f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:22.602817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607ef90-2b23-4710-af72-03fe06e1034f service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:22.609199 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f61c8297-59f4-4c7d-b4ac-7f69eb22123d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:22.610511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f61c8297-59f4-4c7d-b4ac-7f69eb22123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:22.613732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1220/2424] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:22.692111 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1205/2425] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:35:22.725230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607ef90-2b23-4710-af72-03fe06e1034f service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:22.726810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d607ef90-2b23-4710-af72-03fe06e1034f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:22.805368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607ef90-2b23-4710-af72-03fe06e1034f service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:22.869545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208fba9-533a-49b1-917f-6acbc53eb162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:22.870227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208fba9-533a-49b1-917f-6acbc53eb162 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:22.870641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208fba9-533a-49b1-917f-6acbc53eb162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:22.870930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208fba9-533a-49b1-917f-6acbc53eb162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:22.880844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208fba9-533a-49b1-917f-6acbc53eb162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:22.882382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208fba9-533a-49b1-917f-6acbc53eb162 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:22.893463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1221/2426] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:23.175989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c8297-59f4-4c7d-b4ac-7f69eb22123d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:23.177444 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f61c8297-59f4-4c7d-b4ac-7f69eb22123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:23.191143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c8297-59f4-4c7d-b4ac-7f69eb22123d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:23.196002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:23.196901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:23.197191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:23.197467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] PATH_INFO: `/v3/projects/c78029c7de4144d58520e38c17a47dab` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:23.199486 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78029c7de4144d58520e38c17a47dab)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:23.201407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:23.249527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78029c7de4144d58520e38c17a47dab', 'request_id': 'req-d20ae6b1-5230-4e9a-91e0-610c0b343e54'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.257377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78029c7de4144d58520e38c17a47dab', 'request_id': 'req-d20ae6b1-5230-4e9a-91e0-610c0b343e54'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.265702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78029c7de4144d58520e38c17a47dab', 'request_id': 'req-d20ae6b1-5230-4e9a-91e0-610c0b343e54'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.284301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] The token cache is being invalidate because project c78029c7de4144d58520e38c17a47dab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:35:23.284603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.285257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.286860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1206/2427] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:35:23 2022] DELETE /identity/v3/projects/c78029c7de4144d58520e38c17a47dab => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:23.290930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208fba9-533a-49b1-917f-6acbc53eb162 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:23.300973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208fba9-533a-49b1-917f-6acbc53eb162 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:23.364708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c38c3-a3f0-4168-af2c-d824582deef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:23.365603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c38c3-a3f0-4168-af2c-d824582deef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:23.365893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c38c3-a3f0-4168-af2c-d824582deef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:23.366197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c38c3-a3f0-4168-af2c-d824582deef0 admin admin] PATH_INFO: `/v3/users/d162b97496c24118b283e0be3f8d2dd8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:23.367489 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c38c3-a3f0-4168-af2c-d824582deef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d162b97496c24118b283e0be3f8d2dd8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:23.369461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c38c3-a3f0-4168-af2c-d824582deef0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:23.403569 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:23.425016 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5f9c38c3-a3f0-4168-af2c-d824582deef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd162b97496c24118b283e0be3f8d2dd8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.440039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5f9c38c3-a3f0-4168-af2c-d824582deef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd162b97496c24118b283e0be3f8d2dd8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.440789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5f9c38c3-a3f0-4168-af2c-d824582deef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd162b97496c24118b283e0be3f8d2dd8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.449646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5f9c38c3-a3f0-4168-af2c-d824582deef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd162b97496c24118b283e0be3f8d2dd8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.457976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5f9c38c3-a3f0-4168-af2c-d824582deef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd162b97496c24118b283e0be3f8d2dd8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.464578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1222/2428] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:35:23 2022] DELETE /identity/v3/users/d162b97496c24118b283e0be3f8d2dd8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:23.745231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:23.746899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:23.761926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ae6b1-5230-4e9a-91e0-610c0b343e54 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:23.801687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:23.822153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:23.823640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:23.823920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:23.824196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] PATH_INFO: `/v3/projects/b3f81357301b4692a7bbc4fee7e44398` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:23.825392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f81357301b4692a7bbc4fee7e44398)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:23.827363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:23.878678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f81357301b4692a7bbc4fee7e44398', 'request_id': 'req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.886666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f81357301b4692a7bbc4fee7e44398', 'request_id': 'req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.896912 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f81357301b4692a7bbc4fee7e44398', 'request_id': 'req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.912531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] The token cache is being invalidate because project b3f81357301b4692a7bbc4fee7e44398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:35:23.913132 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.913974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-7f33dde7-35e0-41fc-88b2-abaa9ed27e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:23.915797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1207/2429] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:35:23 2022] DELETE /identity/v3/projects/b3f81357301b4692a7bbc4fee7e44398 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:35:23.921148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad0958-e37b-45de-888d-b7daa57deb62 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:23.922072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad0958-e37b-45de-888d-b7daa57deb62 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:23.922486 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bad0958-e37b-45de-888d-b7daa57deb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:23.942053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3bad0958-e37b-45de-888d-b7daa57deb62 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:23.943538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3bad0958-e37b-45de-888d-b7daa57deb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:24.035841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1223/2430] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:35:24.044076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4fe2d-cfa5-47d1-9fd5-aaacbf53553d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:24.044373 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4fe2d-cfa5-47d1-9fd5-aaacbf53553d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:24.044658 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4fe2d-cfa5-47d1-9fd5-aaacbf53553d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:24.063555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a7b4fe2d-cfa5-47d1-9fd5-aaacbf53553d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:24.064783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a7b4fe2d-cfa5-47d1-9fd5-aaacbf53553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:24.073259 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1208/2431] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:24.078879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad0958-e37b-45de-888d-b7daa57deb62 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:24.090098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bad0958-e37b-45de-888d-b7daa57deb62 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:24.094727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e0bcb-6ba8-4fef-8769-edd4adccc512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:24.095394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e0bcb-6ba8-4fef-8769-edd4adccc512 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:24.095677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e0bcb-6ba8-4fef-8769-edd4adccc512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:24.095952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e0bcb-6ba8-4fef-8769-edd4adccc512 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:24.096680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e0bcb-6ba8-4fef-8769-edd4adccc512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:35:24.096987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e0bcb-6ba8-4fef-8769-edd4adccc512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:24.098628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e0bcb-6ba8-4fef-8769-edd4adccc512 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:24.111124 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1224/2432] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:35:24.115574 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4fe2d-cfa5-47d1-9fd5-aaacbf53553d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:24.124848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4fe2d-cfa5-47d1-9fd5-aaacbf53553d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:24.129483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ed4b2-23c2-4df1-b383-c5b350bfd7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:24.130140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ed4b2-23c2-4df1-b383-c5b350bfd7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:24.130425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ed4b2-23c2-4df1-b383-c5b350bfd7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:24.130739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ed4b2-23c2-4df1-b383-c5b350bfd7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:24.131431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ed4b2-23c2-4df1-b383-c5b350bfd7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:24.133335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345ed4b2-23c2-4df1-b383-c5b350bfd7b6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:24.161257 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1209/2433] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:35:24 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:35:24.165304 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e0bcb-6ba8-4fef-8769-edd4adccc512 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:24.175982 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e0bcb-6ba8-4fef-8769-edd4adccc512 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:24.244969 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e0a1d-46bc-47c3-a1d6-e3fa4a1628f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:24.245659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e0a1d-46bc-47c3-a1d6-e3fa4a1628f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:24.245972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e0a1d-46bc-47c3-a1d6-e3fa4a1628f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:24.246254 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e0a1d-46bc-47c3-a1d6-e3fa4a1628f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:24.247012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e0a1d-46bc-47c3-a1d6-e3fa4a1628f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:24.248999 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e0a1d-46bc-47c3-a1d6-e3fa4a1628f0 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:24.270611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1225/2434] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:35:24 2022] POST /identity/v3/users => generated 288 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:35:24.276611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ed4b2-23c2-4df1-b383-c5b350bfd7b6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:24.287337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ed4b2-23c2-4df1-b383-c5b350bfd7b6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:24.292174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3bf07-5227-4221-a07a-7dadd443ed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:24.292858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bf07-5227-4221-a07a-7dadd443ed69 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:24.293148 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bf07-5227-4221-a07a-7dadd443ed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:24.293430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3bf07-5227-4221-a07a-7dadd443ed69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:24.294170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3bf07-5227-4221-a07a-7dadd443ed69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:24.295829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b3bf07-5227-4221-a07a-7dadd443ed69 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:24.311707 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1210/2435] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:35:24.315868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e0a1d-46bc-47c3-a1d6-e3fa4a1628f0 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:24.327789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e0a1d-46bc-47c3-a1d6-e3fa4a1628f0 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:24.332586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37b1d-afc4-4b73-b48b-8f60aaf8e7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:24.333258 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37b1d-afc4-4b73-b48b-8f60aaf8e7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:24.333544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37b1d-afc4-4b73-b48b-8f60aaf8e7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:24.333827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37b1d-afc4-4b73-b48b-8f60aaf8e7c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:24.334569 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc37b1d-afc4-4b73-b48b-8f60aaf8e7c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:24.336218 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc37b1d-afc4-4b73-b48b-8f60aaf8e7c7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:24.347893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1226/2436] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:35:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:35:24.352106 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3bf07-5227-4221-a07a-7dadd443ed69 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:24.363416 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3bf07-5227-4221-a07a-7dadd443ed69 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:24.368138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6264ce-dc37-4eda-b700-9cff32402651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:24.369035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6264ce-dc37-4eda-b700-9cff32402651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:24.369322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6264ce-dc37-4eda-b700-9cff32402651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:24.370186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6264ce-dc37-4eda-b700-9cff32402651 admin admin] PATH_INFO: `/v3/projects/af1fa4326bb34ee08ebc86e605b616c0/users/e92d365b3be648c3aea8e410fc3dad67/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:24.393177 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6264ce-dc37-4eda-b700-9cff32402651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af1fa4326bb34ee08ebc86e605b616c0, user_id=e92d365b3be648c3aea8e410fc3dad67, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:24.398477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6264ce-dc37-4eda-b700-9cff32402651 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:24.414893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1211/2437] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:35:24 2022] PUT /identity/v3/projects/af1fa4326bb34ee08ebc86e605b616c0/users/e92d365b3be648c3aea8e410fc3dad67/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:35:24.425291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f838d8-cf0f-4672-af4c-94bf06ec68a1 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:24.425589 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f838d8-cf0f-4672-af4c-94bf06ec68a1 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:24.425873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f838d8-cf0f-4672-af4c-94bf06ec68a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:24.446472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:24.456530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-62f838d8-cf0f-4672-af4c-94bf06ec68a1 None None] MFA Rules not processed for user `e92d365b3be648c3aea8e410fc3dad67`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:24.458605 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-62f838d8-cf0f-4672-af4c-94bf06ec68a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:24.548131 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1227/2438] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:24.560999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6264ce-dc37-4eda-b700-9cff32402651 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:24.562844 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5b6264ce-dc37-4eda-b700-9cff32402651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:24.631506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6264ce-dc37-4eda-b700-9cff32402651 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:24.635353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea103ba-db7e-47c8-bc07-7e27da6f498f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:24.636022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea103ba-db7e-47c8-bc07-7e27da6f498f service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:24.636311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea103ba-db7e-47c8-bc07-7e27da6f498f service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:24.636586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea103ba-db7e-47c8-bc07-7e27da6f498f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:24.646138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea103ba-db7e-47c8-bc07-7e27da6f498f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:24.647666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea103ba-db7e-47c8-bc07-7e27da6f498f service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:24.658063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1212/2439] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:26.053245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf8514-0b78-4203-a8a0-1992d1dd10b0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:26.053806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf8514-0b78-4203-a8a0-1992d1dd10b0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:26.053806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf8514-0b78-4203-a8a0-1992d1dd10b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:26.072358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a8cf8514-0b78-4203-a8a0-1992d1dd10b0 None None] MFA Rules not processed for user `e92d365b3be648c3aea8e410fc3dad67`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:26.073623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a8cf8514-0b78-4203-a8a0-1992d1dd10b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:26.080932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1228/2440] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:28.754798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6ccb4-2ce0-4447-947b-1a862d782d15 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:28.755310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6ccb4-2ce0-4447-947b-1a862d782d15 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:28.755512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e6ccb4-2ce0-4447-947b-1a862d782d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:28.771802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:28.793860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-61e6ccb4-2ce0-4447-947b-1a862d782d15 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:28.795067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-61e6ccb4-2ce0-4447-947b-1a862d782d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:28.856469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1213/2441] 10.222.0.64 () {58 vars in 967 bytes} [Wed Jul 20 08:35:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:28.910451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf8514-0b78-4203-a8a0-1992d1dd10b0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:28.912020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a8cf8514-0b78-4203-a8a0-1992d1dd10b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:28.994453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf8514-0b78-4203-a8a0-1992d1dd10b0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:28.999970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2613a-4167-4b6d-bf51-c93d65986238 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:29.000630 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2613a-4167-4b6d-bf51-c93d65986238 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:29.000921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2613a-4167-4b6d-bf51-c93d65986238 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:29.001203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2613a-4167-4b6d-bf51-c93d65986238 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:29.021637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2613a-4167-4b6d-bf51-c93d65986238 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:29.021637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2613a-4167-4b6d-bf51-c93d65986238 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:29.030128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1229/2442] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:35:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:35:29.447554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6ccb4-2ce0-4447-947b-1a862d782d15 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:29.449147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-61e6ccb4-2ce0-4447-947b-1a862d782d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:29.535976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e6ccb4-2ce0-4447-947b-1a862d782d15 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:29.540481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e579c8-4ceb-4a0e-b96e-39bfec3e13bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:29.541860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e579c8-4ceb-4a0e-b96e-39bfec3e13bb service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:29.542829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e579c8-4ceb-4a0e-b96e-39bfec3e13bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:29.543129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e579c8-4ceb-4a0e-b96e-39bfec3e13bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:29.545330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d8e579c8-4ceb-4a0e-b96e-39bfec3e13bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:29.640800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e579c8-4ceb-4a0e-b96e-39bfec3e13bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:29.642391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e579c8-4ceb-4a0e-b96e-39bfec3e13bb service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:29.655240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1214/2443] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:37.296930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2613a-4167-4b6d-bf51-c93d65986238 service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:37.298957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8ff2613a-4167-4b6d-bf51-c93d65986238 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:37.385413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e579c8-4ceb-4a0e-b96e-39bfec3e13bb service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:37.387135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d8e579c8-4ceb-4a0e-b96e-39bfec3e13bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:37.391661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2613a-4167-4b6d-bf51-c93d65986238 service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:37.396814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ca2e0-1e19-4893-8eea-546848356cde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:37.397485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ca2e0-1e19-4893-8eea-546848356cde service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:37.397768 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ca2e0-1e19-4893-8eea-546848356cde service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:37.398109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742ca2e0-1e19-4893-8eea-546848356cde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:37.399943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-742ca2e0-1e19-4893-8eea-546848356cde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:37.426071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e579c8-4ceb-4a0e-b96e-39bfec3e13bb service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:37.429089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ca2e0-1e19-4893-8eea-546848356cde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:37.430775 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742ca2e0-1e19-4893-8eea-546848356cde service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:37.438390 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:37.439471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:37.439776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:37.440044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] PATH_INFO: `/v3/users/482e2d3af7994b6692af6a4fa17dde1d` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:37.442164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=482e2d3af7994b6692af6a4fa17dde1d)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:37.445272 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1230/2444] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:37.445802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:37.529600 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '482e2d3af7994b6692af6a4fa17dde1d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:37.531904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:37.542281 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '482e2d3af7994b6692af6a4fa17dde1d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:37.552337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '482e2d3af7994b6692af6a4fa17dde1d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:37.568540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '482e2d3af7994b6692af6a4fa17dde1d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:37.569678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '482e2d3af7994b6692af6a4fa17dde1d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:37.576760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1215/2445] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:35:37 2022] DELETE /identity/v3/users/482e2d3af7994b6692af6a4fa17dde1d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:37.775400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ca2e0-1e19-4893-8eea-546848356cde service placement] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:37.795500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742ca2e0-1e19-4893-8eea-546848356cde service placement] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:37.805808 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:37.871294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:37.872192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:37.872463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:37.872739 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] PATH_INFO: `/v3/projects/5732783c305f47a389104f7ea5ac5fcf` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:37.874038 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5732783c305f47a389104f7ea5ac5fcf)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:37.876075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:37.941801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5732783c305f47a389104f7ea5ac5fcf', 'request_id': 'req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:37.950288 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5732783c305f47a389104f7ea5ac5fcf', 'request_id': 'req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:37.957641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5732783c305f47a389104f7ea5ac5fcf', 'request_id': 'req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:37.970195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] The token cache is being invalidate because project 5732783c305f47a389104f7ea5ac5fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:35:37.970502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:37.971203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:37.972791 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1231/2446] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:35:37 2022] DELETE /identity/v3/projects/5732783c305f47a389104f7ea5ac5fcf => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:37.977950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:37.980095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:37.994810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be6597f-c91e-4251-b3ba-ef79cecba0c4 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:38.069776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:38.071327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:38.071620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:38.071901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] PATH_INFO: `/v3/users/a8f23076a49249048ca9dfc73547c8f2` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:38.074043 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f23076a49249048ca9dfc73547c8f2)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:38.076025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:38.141860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f23076a49249048ca9dfc73547c8f2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:38.150993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f23076a49249048ca9dfc73547c8f2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:38.170902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f23076a49249048ca9dfc73547c8f2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:38.188548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f23076a49249048ca9dfc73547c8f2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:38.189941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f23076a49249048ca9dfc73547c8f2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:38.202269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1216/2447] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:35:37 2022] DELETE /identity/v3/users/a8f23076a49249048ca9dfc73547c8f2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:38.408748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:38.410217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:38.424125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf1841-d01e-45fd-bf8b-f3fce7f9dfc7 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:38.482674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:38.483484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:38.483771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:38.484050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] PATH_INFO: `/v3/projects/d6bb6c3f331a4fb581256890765018ac` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:38.485244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6bb6c3f331a4fb581256890765018ac)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:38.487252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:38.531330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:38.535282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6bb6c3f331a4fb581256890765018ac', 'request_id': 'req-4e540e35-ee7f-40a1-b998-585758b6bc52'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:38.543971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6bb6c3f331a4fb581256890765018ac', 'request_id': 'req-4e540e35-ee7f-40a1-b998-585758b6bc52'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:38.550378 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6bb6c3f331a4fb581256890765018ac', 'request_id': 'req-4e540e35-ee7f-40a1-b998-585758b6bc52'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:38.561560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] The token cache is being invalidate because project d6bb6c3f331a4fb581256890765018ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:35:38.561851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:38.562486 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:38.564055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1232/2448] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:35:38 2022] DELETE /identity/v3/projects/d6bb6c3f331a4fb581256890765018ac => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:41.119835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:41.121656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:41.183926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:41.190941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd584b-8c1e-4344-94cb-12561a4ce653 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:41.253384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4406e-2480-4614-aa99-1735c35aa6fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:41.254244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4406e-2480-4614-aa99-1735c35aa6fb service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:41.254244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4406e-2480-4614-aa99-1735c35aa6fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:41.255179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4406e-2480-4614-aa99-1735c35aa6fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:41.257949 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-11d4406e-2480-4614-aa99-1735c35aa6fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:41.348947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4406e-2480-4614-aa99-1735c35aa6fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:41.350399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4406e-2480-4614-aa99-1735c35aa6fb service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:41.363000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1217/2449] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:42.557810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:42.559360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:42.571790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e540e35-ee7f-40a1-b998-585758b6bc52 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:42.575705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205b3d9-52b7-4f53-93d9-0d86fdc76c49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:42.576354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205b3d9-52b7-4f53-93d9-0d86fdc76c49 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:42.576650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205b3d9-52b7-4f53-93d9-0d86fdc76c49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:42.576933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0205b3d9-52b7-4f53-93d9-0d86fdc76c49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:42.579016 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0205b3d9-52b7-4f53-93d9-0d86fdc76c49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:42.594638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205b3d9-52b7-4f53-93d9-0d86fdc76c49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:42.596115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0205b3d9-52b7-4f53-93d9-0d86fdc76c49 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:42.609437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1233/2450] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:46.677224 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1811e90-5f3f-419a-9d81-efb11aa21418 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:46.677791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1811e90-5f3f-419a-9d81-efb11aa21418 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:46.678178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1811e90-5f3f-419a-9d81-efb11aa21418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:46.698661 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d1811e90-5f3f-419a-9d81-efb11aa21418 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:46.699908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d1811e90-5f3f-419a-9d81-efb11aa21418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:46.758096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1218/2451] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:35:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:35:46.773656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205b3d9-52b7-4f53-93d9-0d86fdc76c49 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:46.775137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0205b3d9-52b7-4f53-93d9-0d86fdc76c49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:46.876379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0205b3d9-52b7-4f53-93d9-0d86fdc76c49 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:46.881702 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98c3b-b120-4861-9108-9af533efe294 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:46.882689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98c3b-b120-4861-9108-9af533efe294 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:46.883018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98c3b-b120-4861-9108-9af533efe294 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:46.883390 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98c3b-b120-4861-9108-9af533efe294 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:46.893993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e98c3b-b120-4861-9108-9af533efe294 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:46.896381 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e98c3b-b120-4861-9108-9af533efe294 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:46.907437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1234/2452] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:35:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:35:47.529752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c60c54-8bf5-493b-97e8-e010ad66b7f8 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:47.530076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c60c54-8bf5-493b-97e8-e010ad66b7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:47.530370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c60c54-8bf5-493b-97e8-e010ad66b7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:47.550855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-56c60c54-8bf5-493b-97e8-e010ad66b7f8 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:47.552183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-56c60c54-8bf5-493b-97e8-e010ad66b7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:47.563408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1219/2453] 10.222.0.64 () {58 vars in 967 bytes} [Wed Jul 20 08:35:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:47.577614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98c3b-b120-4861-9108-9af533efe294 service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:47.590308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98c3b-b120-4861-9108-9af533efe294 service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:47.596877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20569a-51de-406c-a167-259ee1582ced service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:47.597599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20569a-51de-406c-a167-259ee1582ced service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:47.597889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20569a-51de-406c-a167-259ee1582ced service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:47.598232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20569a-51de-406c-a167-259ee1582ced service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:47.607367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20569a-51de-406c-a167-259ee1582ced service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:47.608997 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20569a-51de-406c-a167-259ee1582ced service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:47.618400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1235/2454] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:35:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:35:47.864151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa88b1-d5c1-4673-8298-1b0fef35b451 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:47.864668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa88b1-d5c1-4673-8298-1b0fef35b451 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:47.864808 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa88b1-d5c1-4673-8298-1b0fef35b451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:47.886934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c1aa88b1-d5c1-4673-8298-1b0fef35b451 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:47.888354 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c1aa88b1-d5c1-4673-8298-1b0fef35b451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:47.896642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1220/2455] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:35:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:47.909216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20569a-51de-406c-a167-259ee1582ced service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:47.920382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20569a-51de-406c-a167-259ee1582ced service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:47.929357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b65e7-d2c2-4d80-bf25-e3413021ba84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:47.930010 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b65e7-d2c2-4d80-bf25-e3413021ba84 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:47.930293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b65e7-d2c2-4d80-bf25-e3413021ba84 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:47.930563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b65e7-d2c2-4d80-bf25-e3413021ba84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:47.940736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b65e7-d2c2-4d80-bf25-e3413021ba84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:47.942611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b65e7-d2c2-4d80-bf25-e3413021ba84 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:47.952734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1236/2456] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:35:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 08:35:48.346828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa77cc-a43e-49c3-a8a8-d452de4da744 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:48.347289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa77cc-a43e-49c3-a8a8-d452de4da744 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:48.347672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa77cc-a43e-49c3-a8a8-d452de4da744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:48.367377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-dcfa77cc-a43e-49c3-a8a8-d452de4da744 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:48.368671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dcfa77cc-a43e-49c3-a8a8-d452de4da744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:48.376559 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1221/2457] 10.222.0.64 () {58 vars in 967 bytes} [Wed Jul 20 08:35:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:48.394420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b65e7-d2c2-4d80-bf25-e3413021ba84 service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:48.408690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b65e7-d2c2-4d80-bf25-e3413021ba84 service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:48.414234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb245c21-2e7d-40fc-af7d-76ec1c7f899a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:48.415311 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245c21-2e7d-40fc-af7d-76ec1c7f899a service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:48.415942 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245c21-2e7d-40fc-af7d-76ec1c7f899a service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:48.416934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb245c21-2e7d-40fc-af7d-76ec1c7f899a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:48.427294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb245c21-2e7d-40fc-af7d-76ec1c7f899a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:48.429643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb245c21-2e7d-40fc-af7d-76ec1c7f899a service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:48.441759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1237/2458] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:35:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:35:50.713651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa77cc-a43e-49c3-a8a8-d452de4da744 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:50.715097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dcfa77cc-a43e-49c3-a8a8-d452de4da744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:50.748550 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:50.784867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa77cc-a43e-49c3-a8a8-d452de4da744 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:50.789370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:50.790242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:50.791179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:50.791548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] PATH_INFO: `/v3/users/e92d365b3be648c3aea8e410fc3dad67` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:50.792754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e92d365b3be648c3aea8e410fc3dad67)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:50.794745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:50.841821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e92d365b3be648c3aea8e410fc3dad67'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:50.848733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e92d365b3be648c3aea8e410fc3dad67'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:50.856540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e92d365b3be648c3aea8e410fc3dad67'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:50.868251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e92d365b3be648c3aea8e410fc3dad67'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:50.868942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e92d365b3be648c3aea8e410fc3dad67'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:50.875006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1222/2459] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:35:50 2022] DELETE /identity/v3/users/e92d365b3be648c3aea8e410fc3dad67 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:35:50.879502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3e652-6411-42eb-8408-2a490a062daf None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:50.880003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3e652-6411-42eb-8408-2a490a062daf None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:50.880127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3e652-6411-42eb-8408-2a490a062daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:50.899338 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-71f3e652-6411-42eb-8408-2a490a062daf None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:35:50.901193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-71f3e652-6411-42eb-8408-2a490a062daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:50.962895 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1238/2460] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:35:50.972893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:50.974492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:51.033982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab49c48-eaf2-4f43-9dc7-6dd2ab8ac233 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:51.037581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e423261-1f4c-4512-a50b-4539b98cdfa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:51.038258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e423261-1f4c-4512-a50b-4539b98cdfa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:51.038547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e423261-1f4c-4512-a50b-4539b98cdfa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:51.038871 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e423261-1f4c-4512-a50b-4539b98cdfa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:51.048191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e423261-1f4c-4512-a50b-4539b98cdfa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:51.049653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e423261-1f4c-4512-a50b-4539b98cdfa6 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:51.059339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1223/2461] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:35:51.240836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3e652-6411-42eb-8408-2a490a062daf None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:35:51.243131 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-71f3e652-6411-42eb-8408-2a490a062daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:35:51.265286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3e652-6411-42eb-8408-2a490a062daf None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:35:51.273064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:35:51.274823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:35:51.275525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:35:51.276103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] PATH_INFO: `/v3/projects/af1fa4326bb34ee08ebc86e605b616c0` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:35:51.278178 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af1fa4326bb34ee08ebc86e605b616c0)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:35:51.281970 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:35:51.286487 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:35:51.341833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af1fa4326bb34ee08ebc86e605b616c0', 'request_id': 'req-e32c41c5-0caa-457b-8762-6c2e1b336d04'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:51.350071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af1fa4326bb34ee08ebc86e605b616c0', 'request_id': 'req-e32c41c5-0caa-457b-8762-6c2e1b336d04'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:51.357510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af1fa4326bb34ee08ebc86e605b616c0', 'request_id': 'req-e32c41c5-0caa-457b-8762-6c2e1b336d04'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:51.368169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] The token cache is being invalidate because project af1fa4326bb34ee08ebc86e605b616c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:35:51.369016 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:51.369675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:35:51.371270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1239/2462] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:35:51 2022] DELETE /identity/v3/projects/af1fa4326bb34ee08ebc86e605b616c0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:36:01.054955 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=66,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:36:01.365042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=23,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:36:20.585757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e423261-1f4c-4512-a50b-4539b98cdfa6 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:36:20.588026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4e423261-1f4c-4512-a50b-4539b98cdfa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:20.667475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e423261-1f4c-4512-a50b-4539b98cdfa6 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:36:20.732025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8fbd0-ba32-488b-b44d-eed20d628e7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:36:20.732693 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8fbd0-ba32-488b-b44d-eed20d628e7b service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:36:20.732983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8fbd0-ba32-488b-b44d-eed20d628e7b service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:36:20.733268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8fbd0-ba32-488b-b44d-eed20d628e7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:36:20.735175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cdb8fbd0-ba32-488b-b44d-eed20d628e7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:20.821915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8fbd0-ba32-488b-b44d-eed20d628e7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:36:20.823557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8fbd0-ba32-488b-b44d-eed20d628e7b service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:36:20.843490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1224/2463] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:36:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:36:28.284848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:36:28.286271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:28.303297 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c41c5-0caa-457b-8762-6c2e1b336d04 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:36:28.309648 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c80fa1-6ac6-462b-9506-eb28591d39d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:36:28.310295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c80fa1-6ac6-462b-9506-eb28591d39d2 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:36:28.310598 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c80fa1-6ac6-462b-9506-eb28591d39d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:36:28.310884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c80fa1-6ac6-462b-9506-eb28591d39d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:36:28.312627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-09c80fa1-6ac6-462b-9506-eb28591d39d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:28.395467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c80fa1-6ac6-462b-9506-eb28591d39d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:36:28.397073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c80fa1-6ac6-462b-9506-eb28591d39d2 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:36:28.416720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1240/2464] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:36:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:36:30.840587 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:36:38.413317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=41 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:36:42.572558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e4dd3-332d-44a9-bb6d-6faceaabd546 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:36:42.573492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e4dd3-332d-44a9-bb6d-6faceaabd546 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:36:42.573492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e4dd3-332d-44a9-bb6d-6faceaabd546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:36:42.637815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c26e4dd3-332d-44a9-bb6d-6faceaabd546 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:36:42.639705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c26e4dd3-332d-44a9-bb6d-6faceaabd546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:42.704812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c80fa1-6ac6-462b-9506-eb28591d39d2 service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:36:42.706463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-09c80fa1-6ac6-462b-9506-eb28591d39d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:42.726966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c80fa1-6ac6-462b-9506-eb28591d39d2 service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:36:42.733166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1225/2465] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:36:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:36:42.735214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a51d8d-928d-4b2b-b3ae-68a0c30d69c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:36:42.736184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a51d8d-928d-4b2b-b3ae-68a0c30d69c4 service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:36:42.736697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a51d8d-928d-4b2b-b3ae-68a0c30d69c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:36:42.737215 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a51d8d-928d-4b2b-b3ae-68a0c30d69c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:36:42.740441 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-39a51d8d-928d-4b2b-b3ae-68a0c30d69c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:42.756545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a51d8d-928d-4b2b-b3ae-68a0c30d69c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:36:42.758736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e4dd3-332d-44a9-bb6d-6faceaabd546 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:36:42.758902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a51d8d-928d-4b2b-b3ae-68a0c30d69c4 service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:36:42.760265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c26e4dd3-332d-44a9-bb6d-6faceaabd546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:42.779497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1241/2466] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:36:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:36:42.942064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e4dd3-332d-44a9-bb6d-6faceaabd546 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:36:42.951749 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455ada2-306b-40c7-9598-b5ed89e78062 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:36:42.953386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455ada2-306b-40c7-9598-b5ed89e78062 service swift] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:36:42.953674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455ada2-306b-40c7-9598-b5ed89e78062 service swift] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:36:42.953952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455ada2-306b-40c7-9598-b5ed89e78062 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:36:42.965539 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455ada2-306b-40c7-9598-b5ed89e78062 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:36:42.967451 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455ada2-306b-40c7-9598-b5ed89e78062 service swift] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:36:42.983395 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1226/2467] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:36:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 08:36:46.449214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520f017-4ce2-4318-987a-072b685743d3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:36:46.449824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520f017-4ce2-4318-987a-072b685743d3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:36:46.449824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a520f017-4ce2-4318-987a-072b685743d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:36:46.479230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a520f017-4ce2-4318-987a-072b685743d3 None None] MFA Rules not processed for user `e6c2070c6d2a450c887b8ca6b97c4b38`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:36:46.480497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a520f017-4ce2-4318-987a-072b685743d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:46.487940 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1242/2468] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:36:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:36:46.806255 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455ada2-306b-40c7-9598-b5ed89e78062 service swift] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:36:46.807750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7455ada2-306b-40c7-9598-b5ed89e78062 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:46.862053 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:36:46.874424 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455ada2-306b-40c7-9598-b5ed89e78062 service swift] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:36:46.877988 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f5028-a4ae-4ff0-bd57-198a6c7f63e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:36:46.878684 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f5028-a4ae-4ff0-bd57-198a6c7f63e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:36:46.878976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f5028-a4ae-4ff0-bd57-198a6c7f63e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:36:46.879379 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75f5028-a4ae-4ff0-bd57-198a6c7f63e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:36:46.881090 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f75f5028-a4ae-4ff0-bd57-198a6c7f63e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:46.892001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f5028-a4ae-4ff0-bd57-198a6c7f63e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:36:46.893616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75f5028-a4ae-4ff0-bd57-198a6c7f63e1 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:36:46.904733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1227/2469] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:36:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:36:47.378618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520f017-4ce2-4318-987a-072b685743d3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:36:47.381587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a520f017-4ce2-4318-987a-072b685743d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:47.473497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a520f017-4ce2-4318-987a-072b685743d3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:36:47.479355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b6184-323b-4cec-9d49-2bd52f560f21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:36:47.480412 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b6184-323b-4cec-9d49-2bd52f560f21 service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:36:47.480763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b6184-323b-4cec-9d49-2bd52f560f21 service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:36:47.481109 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b6184-323b-4cec-9d49-2bd52f560f21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:36:47.483085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-861b6184-323b-4cec-9d49-2bd52f560f21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:47.502075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b6184-323b-4cec-9d49-2bd52f560f21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:36:47.504031 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b6184-323b-4cec-9d49-2bd52f560f21 service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:36:47.516452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1243/2470] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:36:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:36:49.433000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f5028-a4ae-4ff0-bd57-198a6c7f63e1 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:36:49.444019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75f5028-a4ae-4ff0-bd57-198a6c7f63e1 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:36:49.454250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa0751-141d-4c15-accc-bd81f9b4b37b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:36:49.455495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa0751-141d-4c15-accc-bd81f9b4b37b service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:36:49.456219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa0751-141d-4c15-accc-bd81f9b4b37b service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:36:49.456219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa0751-141d-4c15-accc-bd81f9b4b37b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:36:49.459536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-82aa0751-141d-4c15-accc-bd81f9b4b37b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:49.478031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa0751-141d-4c15-accc-bd81f9b4b37b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:36:49.479907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa0751-141d-4c15-accc-bd81f9b4b37b service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:36:49.494666 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1228/2471] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:36:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:36:57.511958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=44 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:36:57.662530 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b6184-323b-4cec-9d49-2bd52f560f21 service placement] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:36:57.676152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b6184-323b-4cec-9d49-2bd52f560f21 service placement] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:36:57.681931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc77d7-284e-47e1-b8e7-468aec81f48c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:36:57.682882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc77d7-284e-47e1-b8e7-468aec81f48c service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:36:57.683263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc77d7-284e-47e1-b8e7-468aec81f48c service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:36:57.683656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc77d7-284e-47e1-b8e7-468aec81f48c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:36:57.685532 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-aacc77d7-284e-47e1-b8e7-468aec81f48c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:57.688282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa0751-141d-4c15-accc-bd81f9b4b37b service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:36:57.689677 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-82aa0751-141d-4c15-accc-bd81f9b4b37b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:57.702181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc77d7-284e-47e1-b8e7-468aec81f48c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:36:57.703837 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacc77d7-284e-47e1-b8e7-468aec81f48c service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:36:57.716366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1244/2472] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:36:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:36:57.760174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa0751-141d-4c15-accc-bd81f9b4b37b service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:36:57.764076 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95779af4-c3c0-4dd6-b2c6-4abe1e8fed56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:36:57.764735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95779af4-c3c0-4dd6-b2c6-4abe1e8fed56 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:36:57.765020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95779af4-c3c0-4dd6-b2c6-4abe1e8fed56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:36:57.765297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95779af4-c3c0-4dd6-b2c6-4abe1e8fed56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:36:57.767092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-95779af4-c3c0-4dd6-b2c6-4abe1e8fed56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:36:57.827597 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95779af4-c3c0-4dd6-b2c6-4abe1e8fed56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:36:57.829042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95779af4-c3c0-4dd6-b2c6-4abe1e8fed56 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:36:57.838556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1229/2473] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:36:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:37:01.676214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc77d7-284e-47e1-b8e7-468aec81f48c service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:01.681072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-aacc77d7-284e-47e1-b8e7-468aec81f48c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:01.693008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacc77d7-284e-47e1-b8e7-468aec81f48c service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:01.696928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd47b9-fcee-4afe-84ff-bc60fd470607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:01.697590 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd47b9-fcee-4afe-84ff-bc60fd470607 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:01.697873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd47b9-fcee-4afe-84ff-bc60fd470607 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:01.698145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd47b9-fcee-4afe-84ff-bc60fd470607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:01.699978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dafd47b9-fcee-4afe-84ff-bc60fd470607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:01.783147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd47b9-fcee-4afe-84ff-bc60fd470607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:01.784731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd47b9-fcee-4afe-84ff-bc60fd470607 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:01.797016 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1245/2474] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:37:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:37:02.868829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95779af4-c3c0-4dd6-b2c6-4abe1e8fed56 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:02.890788 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95779af4-c3c0-4dd6-b2c6-4abe1e8fed56 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:02.901402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11302564-5549-4ab5-b27a-b70b750b2063 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:02.902169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11302564-5549-4ab5-b27a-b70b750b2063 service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:02.902526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11302564-5549-4ab5-b27a-b70b750b2063 service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:02.903311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11302564-5549-4ab5-b27a-b70b750b2063 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:02.904841 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-11302564-5549-4ab5-b27a-b70b750b2063 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:02.922771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11302564-5549-4ab5-b27a-b70b750b2063 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:02.924493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11302564-5549-4ab5-b27a-b70b750b2063 service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:02.941720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1230/2475] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:37:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:37:06.608749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd47b9-fcee-4afe-84ff-bc60fd470607 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:06.611701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dafd47b9-fcee-4afe-84ff-bc60fd470607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:06.785150 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd47b9-fcee-4afe-84ff-bc60fd470607 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:06.792193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcca0e6-0a68-46ba-97bd-0f196874eab7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3778997232d44cd3a3c4fb3d53e4db53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:06.792193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcca0e6-0a68-46ba-97bd-0f196874eab7 service glance] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:06.792193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcca0e6-0a68-46ba-97bd-0f196874eab7 service glance] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:06.792718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcca0e6-0a68-46ba-97bd-0f196874eab7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:06.795221 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6fcca0e6-0a68-46ba-97bd-0f196874eab7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:06.818139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcca0e6-0a68-46ba-97bd-0f196874eab7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:06.821813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcca0e6-0a68-46ba-97bd-0f196874eab7 service glance] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:06.838632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1246/2476] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:37:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:37:06.932379 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11302564-5549-4ab5-b27a-b70b750b2063 service nova] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:06.943295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11302564-5549-4ab5-b27a-b70b750b2063 service nova] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:06.947940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3623b2-fe88-4e00-b1c3-3b6ee0d0c702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:06.948781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3623b2-fe88-4e00-b1c3-3b6ee0d0c702 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:06.949172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3623b2-fe88-4e00-b1c3-3b6ee0d0c702 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:06.949551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3623b2-fe88-4e00-b1c3-3b6ee0d0c702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:06.951581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-de3623b2-fe88-4e00-b1c3-3b6ee0d0c702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:06.969664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3623b2-fe88-4e00-b1c3-3b6ee0d0c702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:06.971873 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3623b2-fe88-4e00-b1c3-3b6ee0d0c702 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:06.982887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1231/2477] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:37:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:37:07.094303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcca0e6-0a68-46ba-97bd-0f196874eab7 service glance] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:07.105027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcca0e6-0a68-46ba-97bd-0f196874eab7 service glance] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:07.112864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273b8ac-36ab-478f-9e17-b80f1fc55f1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:07.113413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273b8ac-36ab-478f-9e17-b80f1fc55f1f service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:07.113697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273b8ac-36ab-478f-9e17-b80f1fc55f1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:07.113980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f273b8ac-36ab-478f-9e17-b80f1fc55f1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:07.117513 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f273b8ac-36ab-478f-9e17-b80f1fc55f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:07.135985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273b8ac-36ab-478f-9e17-b80f1fc55f1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:07.138342 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f273b8ac-36ab-478f-9e17-b80f1fc55f1f service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:07.158035 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1247/2478] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:37:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:37:09.235407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c6c2c-26ce-4df5-9b40-f78afc91bc7b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:09.236358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c6c2c-26ce-4df5-9b40-f78afc91bc7b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:09.236919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c6c2c-26ce-4df5-9b40-f78afc91bc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:09.267915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-466c6c2c-26ce-4df5-9b40-f78afc91bc7b None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:37:09.269157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-466c6c2c-26ce-4df5-9b40-f78afc91bc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:09.281963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1232/2479] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Jul 20 08:37:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:37:09.296929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273b8ac-36ab-478f-9e17-b80f1fc55f1f service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:09.319916 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f273b8ac-36ab-478f-9e17-b80f1fc55f1f service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:09.325130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958064e2-ab3f-4fa5-8927-b671ebba3949 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:09.326852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958064e2-ab3f-4fa5-8927-b671ebba3949 service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:09.327330 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958064e2-ab3f-4fa5-8927-b671ebba3949 service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:09.327812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958064e2-ab3f-4fa5-8927-b671ebba3949 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:09.341882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958064e2-ab3f-4fa5-8927-b671ebba3949 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:09.343771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958064e2-ab3f-4fa5-8927-b671ebba3949 service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:09.364181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1248/2480] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:37:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:37:10.148914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694136c5-a6ee-4574-b283-525143b14edc None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:10.149435 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694136c5-a6ee-4574-b283-525143b14edc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:10.149734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694136c5-a6ee-4574-b283-525143b14edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:10.172399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-694136c5-a6ee-4574-b283-525143b14edc None None] MFA Rules not processed for user `19024dea6ca94745b38158beb08f29fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:37:10.173910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-694136c5-a6ee-4574-b283-525143b14edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:10.181618 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1233/2481] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:37:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:37:10.669343 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958064e2-ab3f-4fa5-8927-b671ebba3949 service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:10.684066 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958064e2-ab3f-4fa5-8927-b671ebba3949 service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:10.688455 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac4198-f3ef-485c-914d-f8c111aaa2dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:10.689108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac4198-f3ef-485c-914d-f8c111aaa2dc service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:10.689413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac4198-f3ef-485c-914d-f8c111aaa2dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:10.689750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac4198-f3ef-485c-914d-f8c111aaa2dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:10.692002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0cac4198-f3ef-485c-914d-f8c111aaa2dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:10.704099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac4198-f3ef-485c-914d-f8c111aaa2dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:10.706022 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cac4198-f3ef-485c-914d-f8c111aaa2dc service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:10.716877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1249/2482] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:37:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:37:11.765308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694136c5-a6ee-4574-b283-525143b14edc None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:11.797235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694136c5-a6ee-4574-b283-525143b14edc None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:11.804833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c6494-379b-4e74-8b85-7e91d3c9485a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:11.805796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c6494-379b-4e74-8b85-7e91d3c9485a service nova] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:11.806261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c6494-379b-4e74-8b85-7e91d3c9485a service nova] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:11.806746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c6494-379b-4e74-8b85-7e91d3c9485a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:11.809241 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1f3c6494-379b-4e74-8b85-7e91d3c9485a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:11.872331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:37:11.920631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c6494-379b-4e74-8b85-7e91d3c9485a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:11.922889 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c6494-379b-4e74-8b85-7e91d3c9485a service nova] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:11.962002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1234/2483] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:37:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:37:20.713274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=87 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:37:21.955193 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=20 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:37:28.231072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac4198-f3ef-485c-914d-f8c111aaa2dc service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:28.240544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac4198-f3ef-485c-914d-f8c111aaa2dc service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:28.244537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2755e0c-345a-451f-892e-649375cc854f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:28.245259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755e0c-345a-451f-892e-649375cc854f service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:28.245611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755e0c-345a-451f-892e-649375cc854f service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:28.246565 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755e0c-345a-451f-892e-649375cc854f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:28.247908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e2755e0c-345a-451f-892e-649375cc854f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:28.326769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2755e0c-345a-451f-892e-649375cc854f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:28.328277 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2755e0c-345a-451f-892e-649375cc854f service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:28.341219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1250/2484] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:37:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:37:31.253535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fe8af-ed62-4817-a73e-4396a0f6d9a5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:31.254097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fe8af-ed62-4817-a73e-4396a0f6d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:31.254200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fe8af-ed62-4817-a73e-4396a0f6d9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:31.278246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e57fe8af-ed62-4817-a73e-4396a0f6d9a5 None None] MFA Rules not processed for user `5284fc12641540c3a42f291138b7c069`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:37:31.279634 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e57fe8af-ed62-4817-a73e-4396a0f6d9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:31.289251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1235/2485] 10.222.0.64 () {58 vars in 967 bytes} [Wed Jul 20 08:37:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:37:31.303779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2755e0c-345a-451f-892e-649375cc854f service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:31.316561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2755e0c-345a-451f-892e-649375cc854f service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:31.322885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bc1b9-4add-432d-a5d0-2810b978d5fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3cf87f0515b460dad8c0ff5591c2685', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:31.323857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bc1b9-4add-432d-a5d0-2810b978d5fe service swift] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:31.324290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bc1b9-4add-432d-a5d0-2810b978d5fe service swift] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:31.324792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088bc1b9-4add-432d-a5d0-2810b978d5fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:31.335063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bc1b9-4add-432d-a5d0-2810b978d5fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:31.336901 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088bc1b9-4add-432d-a5d0-2810b978d5fe service swift] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:31.347715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1251/2486] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Jul 20 08:37:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 08:37:36.476254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fe8af-ed62-4817-a73e-4396a0f6d9a5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:36.477874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e57fe8af-ed62-4817-a73e-4396a0f6d9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:36.494210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fe8af-ed62-4817-a73e-4396a0f6d9a5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:36.499043 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:36.500174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:36.500517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:36.500776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] PATH_INFO: `/v3/users/9c3ca706dfed4770a1eef107b28a9233` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:36.502173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c3ca706dfed4770a1eef107b28a9233)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:36.504370 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:36.558802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c3ca706dfed4770a1eef107b28a9233'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:36.565934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c3ca706dfed4770a1eef107b28a9233'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:36.574779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c3ca706dfed4770a1eef107b28a9233'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:36.588038 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c3ca706dfed4770a1eef107b28a9233'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:36.588775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c3ca706dfed4770a1eef107b28a9233'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:36.595077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1236/2487] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:37:36 2022] DELETE /identity/v3/users/9c3ca706dfed4770a1eef107b28a9233 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:37:36.894973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bc1b9-4add-432d-a5d0-2810b978d5fe service swift] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:36.918638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088bc1b9-4add-432d-a5d0-2810b978d5fe service swift] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:36.993660 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:36.994448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:36.994767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:36.995043 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] PATH_INFO: `/v3/projects/714941c1e0034a37b0bf0144c1eac958` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:36.996359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=714941c1e0034a37b0bf0144c1eac958)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:36.998341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:37.056512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714941c1e0034a37b0bf0144c1eac958', 'request_id': 'req-7982a3b7-f35c-4ba0-859a-95f4b5b56078'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.064434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714941c1e0034a37b0bf0144c1eac958', 'request_id': 'req-7982a3b7-f35c-4ba0-859a-95f4b5b56078'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.071921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714941c1e0034a37b0bf0144c1eac958', 'request_id': 'req-7982a3b7-f35c-4ba0-859a-95f4b5b56078'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.084627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] The token cache is being invalidate because project 714941c1e0034a37b0bf0144c1eac958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:37:37.084913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.085579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.087195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1252/2488] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:37:36 2022] DELETE /identity/v3/projects/714941c1e0034a37b0bf0144c1eac958 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:37:37.091405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:37.092931 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:37.107296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14885334-09a6-4ae8-b8c6-5677c257118e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:37.179055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:37.180031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:37.180322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:37.180603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] PATH_INFO: `/v3/users/50d228e676244ec8b45b978665a7b2da` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:37.182012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d228e676244ec8b45b978665a7b2da)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:37.183996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:37.237542 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d228e676244ec8b45b978665a7b2da'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.246031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d228e676244ec8b45b978665a7b2da'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.253687 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d228e676244ec8b45b978665a7b2da'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.258638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:37:37.268510 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d228e676244ec8b45b978665a7b2da'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.269321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d228e676244ec8b45b978665a7b2da'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.275903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1237/2489] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:37:37 2022] DELETE /identity/v3/users/50d228e676244ec8b45b978665a7b2da => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:37:37.474884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:37.476713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:37.491167 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982a3b7-f35c-4ba0-859a-95f4b5b56078 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:37.500144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:37:37.557170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:37.558077 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:37.558442 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:37.558756 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] PATH_INFO: `/v3/projects/c3cf87b84407454091a026745dfe006b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:37.560159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3cf87b84407454091a026745dfe006b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:37.562504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:37.613950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3cf87b84407454091a026745dfe006b', 'request_id': 'req-813d2bfd-eecf-4c8c-95de-a84339f59801'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.637866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3cf87b84407454091a026745dfe006b', 'request_id': 'req-813d2bfd-eecf-4c8c-95de-a84339f59801'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.645762 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3cf87b84407454091a026745dfe006b', 'request_id': 'req-813d2bfd-eecf-4c8c-95de-a84339f59801'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.657775 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] The token cache is being invalidate because project c3cf87b84407454091a026745dfe006b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:37:37.658060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.658776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:37:37.660340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1253/2490] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:37:37 2022] DELETE /identity/v3/projects/c3cf87b84407454091a026745dfe006b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:37:47.272431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=2,keystone:SELECT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:37:47.654223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:37:50.227542 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:37:50.229930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:50.313758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab6ae6-d50b-42d3-b1ab-a7cdff7a254a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:37:50.373931 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036328e-f076-448b-af60-b470d54c3cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:37:50.374822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036328e-f076-448b-af60-b470d54c3cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:37:50.376044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036328e-f076-448b-af60-b470d54c3cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:37:50.376044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036328e-f076-448b-af60-b470d54c3cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:37:50.378483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9036328e-f076-448b-af60-b470d54c3cd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:37:50.475634 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036328e-f076-448b-af60-b470d54c3cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:37:50.477023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036328e-f076-448b-af60-b470d54c3cd6 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:37:50.498152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1238/2491] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:37:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:00.485407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=70 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:01.071786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:01.073314 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:01.087652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d2bfd-eecf-4c8c-95de-a84339f59801 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:01.091594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce71e8-9a89-4620-b565-c7cb99110b80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:01.092255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce71e8-9a89-4620-b565-c7cb99110b80 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:01.092543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce71e8-9a89-4620-b565-c7cb99110b80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:01.092824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ce71e8-9a89-4620-b565-c7cb99110b80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:01.094706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-c2ce71e8-9a89-4620-b565-c7cb99110b80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:01.167633 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce71e8-9a89-4620-b565-c7cb99110b80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:01.169102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ce71e8-9a89-4620-b565-c7cb99110b80 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:01.180748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1254/2492] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:05.010488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036328e-f076-448b-af60-b470d54c3cd6 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:05.012880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-9036328e-f076-448b-af60-b470d54c3cd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:05.035589 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036328e-f076-448b-af60-b470d54c3cd6 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:05.042892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a4a88-1248-4fb1-bf94-19e78d56f49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:05.043980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a4a88-1248-4fb1-bf94-19e78d56f49d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:05.044394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a4a88-1248-4fb1-bf94-19e78d56f49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:05.044816 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a4a88-1248-4fb1-bf94-19e78d56f49d admin admin] PATH_INFO: `/v3/users/b2326ff15f294e15806d8a564770b35e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:05.046308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a4a88-1248-4fb1-bf94-19e78d56f49d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2326ff15f294e15806d8a564770b35e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:05.048755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93a4a88-1248-4fb1-bf94-19e78d56f49d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:05.112533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d93a4a88-1248-4fb1-bf94-19e78d56f49d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2326ff15f294e15806d8a564770b35e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:05.122054 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d93a4a88-1248-4fb1-bf94-19e78d56f49d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2326ff15f294e15806d8a564770b35e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:05.131983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d93a4a88-1248-4fb1-bf94-19e78d56f49d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2326ff15f294e15806d8a564770b35e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:05.146792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d93a4a88-1248-4fb1-bf94-19e78d56f49d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2326ff15f294e15806d8a564770b35e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:05.147659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d93a4a88-1248-4fb1-bf94-19e78d56f49d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2326ff15f294e15806d8a564770b35e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:05.154704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1239/2493] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:05 2022] DELETE /identity/v3/users/b2326ff15f294e15806d8a564770b35e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:38:05.352805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce71e8-9a89-4620-b565-c7cb99110b80 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:05.367052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ce71e8-9a89-4620-b565-c7cb99110b80 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:05.456118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:05.457019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:05.457329 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:05.458796 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] PATH_INFO: `/v3/projects/fa4764ea3c4f4bf98ea56ee0798adfdc` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:05.459522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4764ea3c4f4bf98ea56ee0798adfdc)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:05.461529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:05.522888 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4764ea3c4f4bf98ea56ee0798adfdc', 'request_id': 'req-1ed76b31-9768-4b64-867f-c24396be2a31'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:05.531498 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4764ea3c4f4bf98ea56ee0798adfdc', 'request_id': 'req-1ed76b31-9768-4b64-867f-c24396be2a31'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:05.539563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4764ea3c4f4bf98ea56ee0798adfdc', 'request_id': 'req-1ed76b31-9768-4b64-867f-c24396be2a31'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:05.551451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] The token cache is being invalidate because project fa4764ea3c4f4bf98ea56ee0798adfdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:05.551864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:05.552425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1ed76b31-9768-4b64-867f-c24396be2a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:05.554268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1255/2494] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:05 2022] DELETE /identity/v3/projects/fa4764ea3c4f4bf98ea56ee0798adfdc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:05.559227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a783bff-10e3-45f5-bde7-af5a382e124d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:05.559549 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a783bff-10e3-45f5-bde7-af5a382e124d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:05.559834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a783bff-10e3-45f5-bde7-af5a382e124d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:05.580559 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3a783bff-10e3-45f5-bde7-af5a382e124d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:05.581984 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3a783bff-10e3-45f5-bde7-af5a382e124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:05.651411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1240/2495] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:05.659800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e7a03-de5a-4fff-bfc8-f402068ae442 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:05.660142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e7a03-de5a-4fff-bfc8-f402068ae442 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:05.660861 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e7a03-de5a-4fff-bfc8-f402068ae442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:05.684367 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b33e7a03-de5a-4fff-bfc8-f402068ae442 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:05.685780 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b33e7a03-de5a-4fff-bfc8-f402068ae442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:05.696617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1256/2496] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:38:05.701782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a783bff-10e3-45f5-bde7-af5a382e124d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:05.713758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a783bff-10e3-45f5-bde7-af5a382e124d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:05.719560 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadb523-b1e0-408c-a2e1-ba1b1a218d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:05.720351 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadb523-b1e0-408c-a2e1-ba1b1a218d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:05.720718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadb523-b1e0-408c-a2e1-ba1b1a218d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:05.721140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadb523-b1e0-408c-a2e1-ba1b1a218d4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:05.722146 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadb523-b1e0-408c-a2e1-ba1b1a218d4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:38:05.722533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadb523-b1e0-408c-a2e1-ba1b1a218d4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:05.724502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadb523-b1e0-408c-a2e1-ba1b1a218d4c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:05.740156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1241/2497] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:38:05.744840 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e7a03-de5a-4fff-bfc8-f402068ae442 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:05.755143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e7a03-de5a-4fff-bfc8-f402068ae442 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:05.759921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d61ec4-fae7-4e5b-b0da-824ad43d1ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:05.760589 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d61ec4-fae7-4e5b-b0da-824ad43d1ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:05.760871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d61ec4-fae7-4e5b-b0da-824ad43d1ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:05.761146 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d61ec4-fae7-4e5b-b0da-824ad43d1ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:05.761868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d61ec4-fae7-4e5b-b0da-824ad43d1ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:05.763812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d61ec4-fae7-4e5b-b0da-824ad43d1ebe admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:05.778973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:05.805891 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1257/2498] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:05 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:05.809231 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadb523-b1e0-408c-a2e1-ba1b1a218d4c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:05.820123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadb523-b1e0-408c-a2e1-ba1b1a218d4c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:05.881599 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a3175-2c44-40d3-966c-a7a51e32ad3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:05.882281 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a3175-2c44-40d3-966c-a7a51e32ad3c admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:05.882570 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a3175-2c44-40d3-966c-a7a51e32ad3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:05.882885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272a3175-2c44-40d3-966c-a7a51e32ad3c admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:05.883590 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a3175-2c44-40d3-966c-a7a51e32ad3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:05.885769 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272a3175-2c44-40d3-966c-a7a51e32ad3c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:05.893114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:05.918962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1242/2499] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:05 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:05.923436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d61ec4-fae7-4e5b-b0da-824ad43d1ebe admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:05.935342 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d61ec4-fae7-4e5b-b0da-824ad43d1ebe admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:05.940623 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ebd8a-09a6-41ef-bee0-cfaae8a120c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:05.941321 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ebd8a-09a6-41ef-bee0-cfaae8a120c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:05.941610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ebd8a-09a6-41ef-bee0-cfaae8a120c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:05.942468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ebd8a-09a6-41ef-bee0-cfaae8a120c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:05.943291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ebd8a-09a6-41ef-bee0-cfaae8a120c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:05.944931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ebd8a-09a6-41ef-bee0-cfaae8a120c3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:05.956302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1258/2500] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:05.960462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a3175-2c44-40d3-966c-a7a51e32ad3c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:05.971501 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272a3175-2c44-40d3-966c-a7a51e32ad3c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:05.976996 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63acd88-f448-4dd7-bf0e-e45a48025645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:05.977682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63acd88-f448-4dd7-bf0e-e45a48025645 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:05.978005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63acd88-f448-4dd7-bf0e-e45a48025645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:05.978437 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63acd88-f448-4dd7-bf0e-e45a48025645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:05.979232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63acd88-f448-4dd7-bf0e-e45a48025645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:05.980870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63acd88-f448-4dd7-bf0e-e45a48025645 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:05.993882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1243/2501] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:05.999036 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ebd8a-09a6-41ef-bee0-cfaae8a120c3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:06.013644 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ebd8a-09a6-41ef-bee0-cfaae8a120c3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:06.021602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12070ab9-c628-430e-a82c-c61963a53da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:06.023139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12070ab9-c628-430e-a82c-c61963a53da7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:06.023917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12070ab9-c628-430e-a82c-c61963a53da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:06.023917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12070ab9-c628-430e-a82c-c61963a53da7 admin admin] PATH_INFO: `/v3/projects/58daacd7343e4d94b794af8e8a7ee9e1/users/25d6b199fcf9457cad4ee92766874f10/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:06.044669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12070ab9-c628-430e-a82c-c61963a53da7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58daacd7343e4d94b794af8e8a7ee9e1, user_id=25d6b199fcf9457cad4ee92766874f10, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:06.047483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12070ab9-c628-430e-a82c-c61963a53da7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:06.064612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1259/2502] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:05 2022] PUT /identity/v3/projects/58daacd7343e4d94b794af8e8a7ee9e1/users/25d6b199fcf9457cad4ee92766874f10/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:06.075547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d91c81-a90e-421b-8236-43ea7a43dc22 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:06.075547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d91c81-a90e-421b-8236-43ea7a43dc22 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:06.076251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d91c81-a90e-421b-8236-43ea7a43dc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:06.105220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-75d91c81-a90e-421b-8236-43ea7a43dc22 None None] MFA Rules not processed for user `25d6b199fcf9457cad4ee92766874f10`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:06.106460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-75d91c81-a90e-421b-8236-43ea7a43dc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:06.176562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1244/2503] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:06.181539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12070ab9-c628-430e-a82c-c61963a53da7 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:06.254836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12070ab9-c628-430e-a82c-c61963a53da7 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:06.259767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45e07b-114f-4f3b-a576-6c9fc27c35b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:06.260457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45e07b-114f-4f3b-a576-6c9fc27c35b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:06.260742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45e07b-114f-4f3b-a576-6c9fc27c35b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:06.261030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45e07b-114f-4f3b-a576-6c9fc27c35b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:06.261759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45e07b-114f-4f3b-a576-6c9fc27c35b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:06.263864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45e07b-114f-4f3b-a576-6c9fc27c35b3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:06.296671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1260/2504] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:06 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:06.301455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d91c81-a90e-421b-8236-43ea7a43dc22 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:06.312555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d91c81-a90e-421b-8236-43ea7a43dc22 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:06.390793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad8c47-e8cd-469c-828d-b0a8eae55555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:06.391488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad8c47-e8cd-469c-828d-b0a8eae55555 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:06.391779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad8c47-e8cd-469c-828d-b0a8eae55555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:06.392061 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad8c47-e8cd-469c-828d-b0a8eae55555 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:06.392783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad8c47-e8cd-469c-828d-b0a8eae55555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:06.394856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad8c47-e8cd-469c-828d-b0a8eae55555 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:06.417139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1245/2505] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:06 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:06.421440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45e07b-114f-4f3b-a576-6c9fc27c35b3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:06.432944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45e07b-114f-4f3b-a576-6c9fc27c35b3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:06.437824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca1364-c82f-431e-9f2d-03bc06d1190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:06.438612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca1364-c82f-431e-9f2d-03bc06d1190d admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:06.438905 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca1364-c82f-431e-9f2d-03bc06d1190d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:06.439213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca1364-c82f-431e-9f2d-03bc06d1190d admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:06.439975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca1364-c82f-431e-9f2d-03bc06d1190d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:06.441736 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ca1364-c82f-431e-9f2d-03bc06d1190d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:06.454739 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1261/2506] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:06.459167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad8c47-e8cd-469c-828d-b0a8eae55555 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:06.469790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad8c47-e8cd-469c-828d-b0a8eae55555 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:06.474825 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f66610-9c75-4d5b-a30c-f834541d7b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:06.475718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f66610-9c75-4d5b-a30c-f834541d7b9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:06.476007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f66610-9c75-4d5b-a30c-f834541d7b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:06.476286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f66610-9c75-4d5b-a30c-f834541d7b9e admin admin] PATH_INFO: `/v3/projects/20347d0f172145b29a2b88e353089b04/users/280d577723c64fc9b44ed1a385b7731a/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:06.494161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f66610-9c75-4d5b-a30c-f834541d7b9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=20347d0f172145b29a2b88e353089b04, user_id=280d577723c64fc9b44ed1a385b7731a, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:06.496950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f66610-9c75-4d5b-a30c-f834541d7b9e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:06.512410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1246/2507] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:06 2022] PUT /identity/v3/projects/20347d0f172145b29a2b88e353089b04/users/280d577723c64fc9b44ed1a385b7731a/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:06.517042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca1364-c82f-431e-9f2d-03bc06d1190d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:06.527503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca1364-c82f-431e-9f2d-03bc06d1190d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:06.592908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680bfdb-c855-482e-950a-43d7522d6d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:06.594072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680bfdb-c855-482e-950a-43d7522d6d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:06.594532 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680bfdb-c855-482e-950a-43d7522d6d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:06.594955 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680bfdb-c855-482e-950a-43d7522d6d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:06.595880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680bfdb-c855-482e-950a-43d7522d6d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:06.597882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680bfdb-c855-482e-950a-43d7522d6d2b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:06.609460 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1262/2508] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:06.616594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f66610-9c75-4d5b-a30c-f834541d7b9e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:06.629110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f66610-9c75-4d5b-a30c-f834541d7b9e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:06.635482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da77cc-573c-439c-b883-6c42272976f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:06.636450 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da77cc-573c-439c-b883-6c42272976f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:06.636789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da77cc-573c-439c-b883-6c42272976f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:06.637143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da77cc-573c-439c-b883-6c42272976f6 admin admin] PATH_INFO: `/v3/projects/20347d0f172145b29a2b88e353089b04/users/280d577723c64fc9b44ed1a385b7731a/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:06.639860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da77cc-573c-439c-b883-6c42272976f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20347d0f172145b29a2b88e353089b04, user_id=280d577723c64fc9b44ed1a385b7731a, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:06.642898 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da77cc-573c-439c-b883-6c42272976f6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:06.655553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1247/2509] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:06 2022] PUT /identity/v3/projects/20347d0f172145b29a2b88e353089b04/users/280d577723c64fc9b44ed1a385b7731a/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:06.668560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab1002-9321-410a-a9d2-0f5d1c458e2b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:06.668857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab1002-9321-410a-a9d2-0f5d1c458e2b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:06.669143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab1002-9321-410a-a9d2-0f5d1c458e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:06.703373 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d5ab1002-9321-410a-a9d2-0f5d1c458e2b None None] MFA Rules not processed for user `280d577723c64fc9b44ed1a385b7731a`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:06.704599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d5ab1002-9321-410a-a9d2-0f5d1c458e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:06.714962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:06.790257 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1263/2510] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:06.803112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da77cc-573c-439c-b883-6c42272976f6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:06.805049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e8da77cc-573c-439c-b883-6c42272976f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:06.830194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:06.871194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da77cc-573c-439c-b883-6c42272976f6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:06.875322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c320a1-fa44-4f8e-a968-400cd4b6e326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:06.876133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c320a1-fa44-4f8e-a968-400cd4b6e326 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:06.876278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c320a1-fa44-4f8e-a968-400cd4b6e326 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:06.876558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c320a1-fa44-4f8e-a968-400cd4b6e326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:06.887323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c320a1-fa44-4f8e-a968-400cd4b6e326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:06.888803 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c320a1-fa44-4f8e-a968-400cd4b6e326 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:06.900168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1248/2511] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:07.355385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab1002-9321-410a-a9d2-0f5d1c458e2b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:07.424063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab1002-9321-410a-a9d2-0f5d1c458e2b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:07.431123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:07.432018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:07.432325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:07.432602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] PATH_INFO: `/v3/users/25d6b199fcf9457cad4ee92766874f10` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:07.433820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d6b199fcf9457cad4ee92766874f10)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:07.435899 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:07.491996 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d6b199fcf9457cad4ee92766874f10'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:07.505126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d6b199fcf9457cad4ee92766874f10'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:07.505953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d6b199fcf9457cad4ee92766874f10'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:07.516193 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d6b199fcf9457cad4ee92766874f10'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:07.525102 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d6b199fcf9457cad4ee92766874f10'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:07.532103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1264/2512] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:07 2022] DELETE /identity/v3/users/25d6b199fcf9457cad4ee92766874f10 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:07.536909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598355e-5485-4f28-8d08-b944b0db6cab None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:07.537443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598355e-5485-4f28-8d08-b944b0db6cab None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:07.537443 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598355e-5485-4f28-8d08-b944b0db6cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:07.556747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-5598355e-5485-4f28-8d08-b944b0db6cab None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:07.558040 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5598355e-5485-4f28-8d08-b944b0db6cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:07.625020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1249/2513] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:07.636025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:07.637748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:07.673223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:07.704921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447754ed-8528-467e-a135-b9f32c06cdd2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:07.709525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66566d7d-e1cf-4782-9a44-664508f0166a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:07.710185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66566d7d-e1cf-4782-9a44-664508f0166a service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:07.710472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66566d7d-e1cf-4782-9a44-664508f0166a service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:07.710785 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66566d7d-e1cf-4782-9a44-664508f0166a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:07.720552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66566d7d-e1cf-4782-9a44-664508f0166a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:07.722069 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66566d7d-e1cf-4782-9a44-664508f0166a service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:07.733410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1265/2514] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:08.037942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598355e-5485-4f28-8d08-b944b0db6cab None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:08.040176 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5598355e-5485-4f28-8d08-b944b0db6cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:08.063123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598355e-5485-4f28-8d08-b944b0db6cab None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:08.071799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:08.074713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:08.075248 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:08.075766 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] PATH_INFO: `/v3/projects/58daacd7343e4d94b794af8e8a7ee9e1` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:08.078082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58daacd7343e4d94b794af8e8a7ee9e1)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:08.082015 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:08.152455 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58daacd7343e4d94b794af8e8a7ee9e1', 'request_id': 'req-8a2231e0-1a2e-4c45-9f14-407361eb9709'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.158045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58daacd7343e4d94b794af8e8a7ee9e1', 'request_id': 'req-8a2231e0-1a2e-4c45-9f14-407361eb9709'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.169275 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58daacd7343e4d94b794af8e8a7ee9e1', 'request_id': 'req-8a2231e0-1a2e-4c45-9f14-407361eb9709'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.188840 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] The token cache is being invalidate because project 58daacd7343e4d94b794af8e8a7ee9e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:08.189226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.190047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.192990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1250/2515] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:08 2022] DELETE /identity/v3/projects/58daacd7343e4d94b794af8e8a7ee9e1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:08.197216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66566d7d-e1cf-4782-9a44-664508f0166a service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:08.207170 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66566d7d-e1cf-4782-9a44-664508f0166a service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:08.267599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5aa43-3328-4d9f-93c4-473f67b98419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:08.268483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5aa43-3328-4d9f-93c4-473f67b98419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:08.268764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5aa43-3328-4d9f-93c4-473f67b98419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:08.269041 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5aa43-3328-4d9f-93c4-473f67b98419 admin admin] PATH_INFO: `/v3/users/280d577723c64fc9b44ed1a385b7731a` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:08.270524 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5aa43-3328-4d9f-93c4-473f67b98419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=280d577723c64fc9b44ed1a385b7731a)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:08.272750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5aa43-3328-4d9f-93c4-473f67b98419 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:08.323293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fcb5aa43-3328-4d9f-93c4-473f67b98419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280d577723c64fc9b44ed1a385b7731a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.335500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fcb5aa43-3328-4d9f-93c4-473f67b98419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280d577723c64fc9b44ed1a385b7731a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.336308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fcb5aa43-3328-4d9f-93c4-473f67b98419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280d577723c64fc9b44ed1a385b7731a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.344932 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fcb5aa43-3328-4d9f-93c4-473f67b98419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280d577723c64fc9b44ed1a385b7731a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.353259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-fcb5aa43-3328-4d9f-93c4-473f67b98419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280d577723c64fc9b44ed1a385b7731a'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.359391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1266/2516] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:08 2022] DELETE /identity/v3/users/280d577723c64fc9b44ed1a385b7731a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:08.655449 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:08.658627 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:08.673893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2231e0-1a2e-4c45-9f14-407361eb9709 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:08.720303 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:08.735467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:08.736341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:08.736656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:08.736937 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] PATH_INFO: `/v3/projects/20347d0f172145b29a2b88e353089b04` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:08.738117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] RBAC: Authorizing `identity:delete_project(project_id=20347d0f172145b29a2b88e353089b04)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:08.740138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:08.788952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20347d0f172145b29a2b88e353089b04', 'request_id': 'req-81a3f0be-16bc-4b06-9a14-3712d9a9918e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.794437 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20347d0f172145b29a2b88e353089b04', 'request_id': 'req-81a3f0be-16bc-4b06-9a14-3712d9a9918e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.800591 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20347d0f172145b29a2b88e353089b04', 'request_id': 'req-81a3f0be-16bc-4b06-9a14-3712d9a9918e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.813668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] The token cache is being invalidate because project 20347d0f172145b29a2b88e353089b04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:08.813957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.814640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-81a3f0be-16bc-4b06-9a14-3712d9a9918e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:08.816217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1251/2517] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:08 2022] DELETE /identity/v3/projects/20347d0f172145b29a2b88e353089b04 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:08.821481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c892d-44b0-4103-b8e2-f0f7f1698a78 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:08.821953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c892d-44b0-4103-b8e2-f0f7f1698a78 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:08.821953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c892d-44b0-4103-b8e2-f0f7f1698a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:08.842385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-340c892d-44b0-4103-b8e2-f0f7f1698a78 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:08.843625 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-340c892d-44b0-4103-b8e2-f0f7f1698a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:08.888012 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:08.906695 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1267/2518] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:08.915274 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd964b-3450-4921-b22a-1a13105c75d9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:08.915565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd964b-3450-4921-b22a-1a13105c75d9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:08.915843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd964b-3450-4921-b22a-1a13105c75d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:08.934519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d8dd964b-3450-4921-b22a-1a13105c75d9 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:08.935767 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d8dd964b-3450-4921-b22a-1a13105c75d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:08.944175 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1252/2519] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:08.948703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c892d-44b0-4103-b8e2-f0f7f1698a78 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:08.959855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c892d-44b0-4103-b8e2-f0f7f1698a78 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:08.965793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d939c94-a916-4d03-a79c-e3f497e5524d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:08.966435 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d939c94-a916-4d03-a79c-e3f497e5524d admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:08.966839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d939c94-a916-4d03-a79c-e3f497e5524d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:08.967134 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d939c94-a916-4d03-a79c-e3f497e5524d admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:08.967975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d939c94-a916-4d03-a79c-e3f497e5524d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:38:08.968289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d939c94-a916-4d03-a79c-e3f497e5524d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:08.970030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d939c94-a916-4d03-a79c-e3f497e5524d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:08.982690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1268/2520] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:38:08.987400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd964b-3450-4921-b22a-1a13105c75d9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:08.997148 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd964b-3450-4921-b22a-1a13105c75d9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:09.001806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed2f07-07a6-41d5-b67b-bfc89ab93706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:09.002461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed2f07-07a6-41d5-b67b-bfc89ab93706 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.002810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed2f07-07a6-41d5-b67b-bfc89ab93706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.003073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed2f07-07a6-41d5-b67b-bfc89ab93706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.003775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed2f07-07a6-41d5-b67b-bfc89ab93706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:09.006476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed2f07-07a6-41d5-b67b-bfc89ab93706 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:09.037464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1253/2521] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:08 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:09.041760 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d939c94-a916-4d03-a79c-e3f497e5524d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:09.053111 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d939c94-a916-4d03-a79c-e3f497e5524d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:09.113086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d052a-ba8f-4daa-8dd4-bcd3ebfe310d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:09.113913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d052a-ba8f-4daa-8dd4-bcd3ebfe310d admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.114203 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d052a-ba8f-4daa-8dd4-bcd3ebfe310d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.114478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d052a-ba8f-4daa-8dd4-bcd3ebfe310d admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.115228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d052a-ba8f-4daa-8dd4-bcd3ebfe310d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:09.117235 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d052a-ba8f-4daa-8dd4-bcd3ebfe310d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:09.140293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1269/2522] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:09 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:09.145430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed2f07-07a6-41d5-b67b-bfc89ab93706 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:09.156209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed2f07-07a6-41d5-b67b-bfc89ab93706 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:09.165638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a531e-f059-4dfd-be1b-9726287a59a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:09.166465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a531e-f059-4dfd-be1b-9726287a59a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.166935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a531e-f059-4dfd-be1b-9726287a59a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.167281 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a531e-f059-4dfd-be1b-9726287a59a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.168075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a531e-f059-4dfd-be1b-9726287a59a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:09.169758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a531e-f059-4dfd-be1b-9726287a59a8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:09.181810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1254/2523] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:09.187023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d052a-ba8f-4daa-8dd4-bcd3ebfe310d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:09.198432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d052a-ba8f-4daa-8dd4-bcd3ebfe310d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:09.203908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae39f6-8589-4da2-8231-60c54b6d5e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:09.204574 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae39f6-8589-4da2-8231-60c54b6d5e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.204855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae39f6-8589-4da2-8231-60c54b6d5e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.205128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae39f6-8589-4da2-8231-60c54b6d5e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.205944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae39f6-8589-4da2-8231-60c54b6d5e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:09.208212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae39f6-8589-4da2-8231-60c54b6d5e59 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:09.220056 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1270/2524] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:09.224730 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a531e-f059-4dfd-be1b-9726287a59a8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:09.236385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a531e-f059-4dfd-be1b-9726287a59a8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:09.241929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a56bc-8bee-44bf-907d-e3b9bcd7313b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:09.242890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a56bc-8bee-44bf-907d-e3b9bcd7313b admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.243259 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a56bc-8bee-44bf-907d-e3b9bcd7313b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.243469 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a56bc-8bee-44bf-907d-e3b9bcd7313b admin admin] PATH_INFO: `/v3/projects/3f9345b8750944448ac9bab7b9ece282/users/f8becf0c138b4e4eafaf673bb233c0b1/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.261009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a56bc-8bee-44bf-907d-e3b9bcd7313b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f9345b8750944448ac9bab7b9ece282, user_id=f8becf0c138b4e4eafaf673bb233c0b1, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:09.263822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a56bc-8bee-44bf-907d-e3b9bcd7313b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:09.281128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1255/2525] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:09 2022] PUT /identity/v3/projects/3f9345b8750944448ac9bab7b9ece282/users/f8becf0c138b4e4eafaf673bb233c0b1/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:38:09.292866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc3f88-da00-4749-92c9-8af00149bd27 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.293159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc3f88-da00-4749-92c9-8af00149bd27 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.293448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc3f88-da00-4749-92c9-8af00149bd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.326692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6bfc3f88-da00-4749-92c9-8af00149bd27 None None] MFA Rules not processed for user `f8becf0c138b4e4eafaf673bb233c0b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:09.327938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6bfc3f88-da00-4749-92c9-8af00149bd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:09.399863 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1271/2526] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:09.404621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a56bc-8bee-44bf-907d-e3b9bcd7313b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:09.468020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a56bc-8bee-44bf-907d-e3b9bcd7313b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:09.472962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10305c-5c6f-4ea9-a576-446c8edb3c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:09.473641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10305c-5c6f-4ea9-a576-446c8edb3c4a admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.473940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10305c-5c6f-4ea9-a576-446c8edb3c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.474239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10305c-5c6f-4ea9-a576-446c8edb3c4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.474998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10305c-5c6f-4ea9-a576-446c8edb3c4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:09.476936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10305c-5c6f-4ea9-a576-446c8edb3c4a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:09.499561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:09.510660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1256/2527] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:09 2022] POST /identity/v3/projects => generated 389 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:09.513924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc3f88-da00-4749-92c9-8af00149bd27 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:09.525495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc3f88-da00-4749-92c9-8af00149bd27 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:09.574499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:09.594122 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb457d-ea5b-4429-bf12-0bd167f004dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:09.594865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb457d-ea5b-4429-bf12-0bd167f004dc admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.595151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb457d-ea5b-4429-bf12-0bd167f004dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.595434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb457d-ea5b-4429-bf12-0bd167f004dc admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.596135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb457d-ea5b-4429-bf12-0bd167f004dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:09.598381 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb457d-ea5b-4429-bf12-0bd167f004dc admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:09.621797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1272/2528] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:09 2022] POST /identity/v3/users => generated 301 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:09.626036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10305c-5c6f-4ea9-a576-446c8edb3c4a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:09.637727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10305c-5c6f-4ea9-a576-446c8edb3c4a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:09.642902 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963f9de-44d4-4bed-a1fc-d64d702e43c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:09.643586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963f9de-44d4-4bed-a1fc-d64d702e43c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.643875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963f9de-44d4-4bed-a1fc-d64d702e43c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.644156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963f9de-44d4-4bed-a1fc-d64d702e43c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.645046 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963f9de-44d4-4bed-a1fc-d64d702e43c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:09.646745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963f9de-44d4-4bed-a1fc-d64d702e43c5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:09.659155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1257/2529] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:38:09.665424 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb457d-ea5b-4429-bf12-0bd167f004dc admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:09.677428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb457d-ea5b-4429-bf12-0bd167f004dc admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:09.682401 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4db80-cf38-4d80-91ca-18cd5caa7ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:09.683740 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4db80-cf38-4d80-91ca-18cd5caa7ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.684135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4db80-cf38-4d80-91ca-18cd5caa7ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.684507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4db80-cf38-4d80-91ca-18cd5caa7ea8 admin admin] PATH_INFO: `/v3/projects/96a43d6f99f44fd19f85a539d68ce7bb/users/7d1c689f33a54b8c8b937f68510ff401/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.703275 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4db80-cf38-4d80-91ca-18cd5caa7ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a43d6f99f44fd19f85a539d68ce7bb, user_id=7d1c689f33a54b8c8b937f68510ff401, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:09.706270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4db80-cf38-4d80-91ca-18cd5caa7ea8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:09.722446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1273/2530] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:09 2022] PUT /identity/v3/projects/96a43d6f99f44fd19f85a539d68ce7bb/users/7d1c689f33a54b8c8b937f68510ff401/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:09.728200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963f9de-44d4-4bed-a1fc-d64d702e43c5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:09.739904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963f9de-44d4-4bed-a1fc-d64d702e43c5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:09.807707 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81521eee-d305-4955-9217-6bebb9050bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:09.808383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81521eee-d305-4955-9217-6bebb9050bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.808670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81521eee-d305-4955-9217-6bebb9050bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.808950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81521eee-d305-4955-9217-6bebb9050bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.809765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81521eee-d305-4955-9217-6bebb9050bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:09.811432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81521eee-d305-4955-9217-6bebb9050bd3 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:09.825983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1258/2531] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:09.830474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4db80-cf38-4d80-91ca-18cd5caa7ea8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:09.841467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4db80-cf38-4d80-91ca-18cd5caa7ea8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:09.846683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3944a7-9be2-4fa0-8aa7-4cea19a2f1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:09.847581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3944a7-9be2-4fa0-8aa7-4cea19a2f1ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.848377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3944a7-9be2-4fa0-8aa7-4cea19a2f1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.848654 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3944a7-9be2-4fa0-8aa7-4cea19a2f1ea admin admin] PATH_INFO: `/v3/projects/96a43d6f99f44fd19f85a539d68ce7bb/users/7d1c689f33a54b8c8b937f68510ff401/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.850979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3944a7-9be2-4fa0-8aa7-4cea19a2f1ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=96a43d6f99f44fd19f85a539d68ce7bb, user_id=7d1c689f33a54b8c8b937f68510ff401, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:09.854315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3944a7-9be2-4fa0-8aa7-4cea19a2f1ea admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:09.872478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1274/2532] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:09 2022] PUT /identity/v3/projects/96a43d6f99f44fd19f85a539d68ce7bb/users/7d1c689f33a54b8c8b937f68510ff401/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:09.886564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e84ad2-3458-4454-9058-4c6ff85d292b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:09.886941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e84ad2-3458-4454-9058-4c6ff85d292b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:09.887170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e84ad2-3458-4454-9058-4c6ff85d292b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:09.917647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b3e84ad2-3458-4454-9058-4c6ff85d292b None None] MFA Rules not processed for user `7d1c689f33a54b8c8b937f68510ff401`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:09.918850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b3e84ad2-3458-4454-9058-4c6ff85d292b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:09.996572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1259/2533] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:09 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:10.007835 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3944a7-9be2-4fa0-8aa7-4cea19a2f1ea admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:10.009600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dd3944a7-9be2-4fa0-8aa7-4cea19a2f1ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:10.076911 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3944a7-9be2-4fa0-8aa7-4cea19a2f1ea admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:10.080703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e873ddef-68db-4238-a4ea-9a905728ce6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:10.081362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873ddef-68db-4238-a4ea-9a905728ce6c service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:10.081650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873ddef-68db-4238-a4ea-9a905728ce6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:10.081931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e873ddef-68db-4238-a4ea-9a905728ce6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:10.095160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873ddef-68db-4238-a4ea-9a905728ce6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:10.096878 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e873ddef-68db-4238-a4ea-9a905728ce6c service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:10.108591 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1275/2534] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:10.253747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e84ad2-3458-4454-9058-4c6ff85d292b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:10.327266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e84ad2-3458-4454-9058-4c6ff85d292b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:10.333082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:10.334138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:10.334427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:10.334748 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] PATH_INFO: `/v3/users/f8becf0c138b4e4eafaf673bb233c0b1` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:10.335993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8becf0c138b4e4eafaf673bb233c0b1)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:10.337945 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:10.351397 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=100 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:10.399291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8becf0c138b4e4eafaf673bb233c0b1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:10.406834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8becf0c138b4e4eafaf673bb233c0b1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:10.415696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8becf0c138b4e4eafaf673bb233c0b1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:10.430339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8becf0c138b4e4eafaf673bb233c0b1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:10.431190 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8becf0c138b4e4eafaf673bb233c0b1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:10.438285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1260/2535] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:10 2022] DELETE /identity/v3/users/f8becf0c138b4e4eafaf673bb233c0b1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:10.443761 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82445123-2aee-40bb-bd7e-8700e88ad4b7 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:10.445053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82445123-2aee-40bb-bd7e-8700e88ad4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:10.445337 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82445123-2aee-40bb-bd7e-8700e88ad4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:10.465710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-82445123-2aee-40bb-bd7e-8700e88ad4b7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:10.466973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-82445123-2aee-40bb-bd7e-8700e88ad4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:10.531465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1276/2536] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:10.542073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:10.543574 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:10.608005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14a9f8-75bf-4337-a8d6-198c8121e7be admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:10.611976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8e4a2-9c34-4026-a632-67f6541d7073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:10.612635 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8e4a2-9c34-4026-a632-67f6541d7073 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:10.612926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8e4a2-9c34-4026-a632-67f6541d7073 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:10.613238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8e4a2-9c34-4026-a632-67f6541d7073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:10.623792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8e4a2-9c34-4026-a632-67f6541d7073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:10.625339 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8e4a2-9c34-4026-a632-67f6541d7073 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:10.636863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1261/2537] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:10.945029 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82445123-2aee-40bb-bd7e-8700e88ad4b7 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:10.946721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-82445123-2aee-40bb-bd7e-8700e88ad4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:10.965982 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82445123-2aee-40bb-bd7e-8700e88ad4b7 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:10.972023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:10.974961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:10.975465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:10.976048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] PATH_INFO: `/v3/projects/3f9345b8750944448ac9bab7b9ece282` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:10.977710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f9345b8750944448ac9bab7b9ece282)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:10.980068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:11.029979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:11.046067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f9345b8750944448ac9bab7b9ece282', 'request_id': 'req-72f24e6e-9551-421b-9f2d-817ce6500852'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.055672 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f9345b8750944448ac9bab7b9ece282', 'request_id': 'req-72f24e6e-9551-421b-9f2d-817ce6500852'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.062880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f9345b8750944448ac9bab7b9ece282', 'request_id': 'req-72f24e6e-9551-421b-9f2d-817ce6500852'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.075278 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] The token cache is being invalidate because project 3f9345b8750944448ac9bab7b9ece282 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:11.075581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.076315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.078040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1277/2538] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:10 2022] DELETE /identity/v3/projects/3f9345b8750944448ac9bab7b9ece282 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:11.082413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8e4a2-9c34-4026-a632-67f6541d7073 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:11.093678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8e4a2-9c34-4026-a632-67f6541d7073 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:11.155092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc381135-9290-496c-89c5-3b929c03b5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:11.155991 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc381135-9290-496c-89c5-3b929c03b5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:11.156285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc381135-9290-496c-89c5-3b929c03b5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:11.156566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc381135-9290-496c-89c5-3b929c03b5a8 admin admin] PATH_INFO: `/v3/users/7d1c689f33a54b8c8b937f68510ff401` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:11.157822 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc381135-9290-496c-89c5-3b929c03b5a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1c689f33a54b8c8b937f68510ff401)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:11.159813 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc381135-9290-496c-89c5-3b929c03b5a8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:11.200900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:11.217950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bc381135-9290-496c-89c5-3b929c03b5a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1c689f33a54b8c8b937f68510ff401'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.226057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bc381135-9290-496c-89c5-3b929c03b5a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1c689f33a54b8c8b937f68510ff401'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.234884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bc381135-9290-496c-89c5-3b929c03b5a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1c689f33a54b8c8b937f68510ff401'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.249620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bc381135-9290-496c-89c5-3b929c03b5a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1c689f33a54b8c8b937f68510ff401'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.250365 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-bc381135-9290-496c-89c5-3b929c03b5a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1c689f33a54b8c8b937f68510ff401'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.256892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1262/2539] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:11 2022] DELETE /identity/v3/users/7d1c689f33a54b8c8b937f68510ff401 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:11.609912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:11.611454 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:11.625746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f24e6e-9551-421b-9f2d-817ce6500852 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:11.691045 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:11.691965 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:11.692255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:11.692563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] PATH_INFO: `/v3/projects/96a43d6f99f44fd19f85a539d68ce7bb` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:11.693887 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96a43d6f99f44fd19f85a539d68ce7bb)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:11.695863 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:11.752867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a43d6f99f44fd19f85a539d68ce7bb', 'request_id': 'req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.762014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a43d6f99f44fd19f85a539d68ce7bb', 'request_id': 'req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.770754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a43d6f99f44fd19f85a539d68ce7bb', 'request_id': 'req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.782712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] The token cache is being invalidate because project 96a43d6f99f44fd19f85a539d68ce7bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:11.783007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.783683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-1dbe1a25-2aa1-45eb-a328-a322c9d84db6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:11.786458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1278/2540] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:11 2022] DELETE /identity/v3/projects/96a43d6f99f44fd19f85a539d68ce7bb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:38:11.791860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96366508-42e8-4872-8814-923bd7a850c4 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:11.792157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96366508-42e8-4872-8814-923bd7a850c4 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:11.792445 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96366508-42e8-4872-8814-923bd7a850c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:11.813887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-96366508-42e8-4872-8814-923bd7a850c4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:11.815153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-96366508-42e8-4872-8814-923bd7a850c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:11.880828 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1263/2541] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:11.890184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f2557-383a-4143-9053-e22cda11838d None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:11.890617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f2557-383a-4143-9053-e22cda11838d None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:11.890776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f2557-383a-4143-9053-e22cda11838d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:11.911415 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-826f2557-383a-4143-9053-e22cda11838d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:11.912674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-826f2557-383a-4143-9053-e22cda11838d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:11.921024 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1279/2542] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:38:11.925972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96366508-42e8-4872-8814-923bd7a850c4 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:11.936380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96366508-42e8-4872-8814-923bd7a850c4 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:11.942646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7041ba3-07a5-4d53-901f-4b0c451d1413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:11.943353 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7041ba3-07a5-4d53-901f-4b0c451d1413 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:11.943633 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7041ba3-07a5-4d53-901f-4b0c451d1413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:11.943914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7041ba3-07a5-4d53-901f-4b0c451d1413 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:11.944661 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7041ba3-07a5-4d53-901f-4b0c451d1413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:38:11.944976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7041ba3-07a5-4d53-901f-4b0c451d1413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:11.946728 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7041ba3-07a5-4d53-901f-4b0c451d1413 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:11.968209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1264/2543] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:38:11.973114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f2557-383a-4143-9053-e22cda11838d None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:11.983201 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f2557-383a-4143-9053-e22cda11838d None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:11.987938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9af50-599c-4692-98c2-45e78640f1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:11.988636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9af50-599c-4692-98c2-45e78640f1d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:11.988922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9af50-599c-4692-98c2-45e78640f1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:11.989227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9af50-599c-4692-98c2-45e78640f1d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:11.989936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9af50-599c-4692-98c2-45e78640f1d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:11.991929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9af50-599c-4692-98c2-45e78640f1d4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:12.023174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1280/2544] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:11 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:12.028230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7041ba3-07a5-4d53-901f-4b0c451d1413 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:12.038554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7041ba3-07a5-4d53-901f-4b0c451d1413 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:12.106305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ff676-d9a3-4516-a297-1def9f7b2c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:12.107027 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ff676-d9a3-4516-a297-1def9f7b2c60 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:12.107292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ff676-d9a3-4516-a297-1def9f7b2c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:12.107565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ff676-d9a3-4516-a297-1def9f7b2c60 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:12.108307 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ff676-d9a3-4516-a297-1def9f7b2c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:12.110376 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ff676-d9a3-4516-a297-1def9f7b2c60 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:12.132533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1265/2545] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:12 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:38:12.136715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9af50-599c-4692-98c2-45e78640f1d4 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:12.148585 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9af50-599c-4692-98c2-45e78640f1d4 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:12.153775 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31fcba-ba8c-4c46-959e-2398c20b576d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:12.154458 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31fcba-ba8c-4c46-959e-2398c20b576d admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:12.154791 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31fcba-ba8c-4c46-959e-2398c20b576d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:12.155061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31fcba-ba8c-4c46-959e-2398c20b576d admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:12.155816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31fcba-ba8c-4c46-959e-2398c20b576d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:12.157466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31fcba-ba8c-4c46-959e-2398c20b576d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:12.170519 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1281/2546] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:38:12.174975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ff676-d9a3-4516-a297-1def9f7b2c60 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:12.189520 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ff676-d9a3-4516-a297-1def9f7b2c60 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:12.195325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b0ac0-fdfe-48b2-a7d0-1b6d83ee5cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:12.196182 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b0ac0-fdfe-48b2-a7d0-1b6d83ee5cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:12.196582 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b0ac0-fdfe-48b2-a7d0-1b6d83ee5cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:12.196997 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b0ac0-fdfe-48b2-a7d0-1b6d83ee5cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:12.197876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b0ac0-fdfe-48b2-a7d0-1b6d83ee5cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:12.199936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b0ac0-fdfe-48b2-a7d0-1b6d83ee5cb4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:12.218502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1266/2547] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:38:12.221913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31fcba-ba8c-4c46-959e-2398c20b576d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:12.239434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31fcba-ba8c-4c46-959e-2398c20b576d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:12.247152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e87bb0-a46c-4d3e-a95b-ee4fbc891569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:12.248976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e87bb0-a46c-4d3e-a95b-ee4fbc891569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:12.249438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e87bb0-a46c-4d3e-a95b-ee4fbc891569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:12.250011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e87bb0-a46c-4d3e-a95b-ee4fbc891569 admin admin] PATH_INFO: `/v3/projects/55702662947b4f8ab4f9533b9d0172bc/users/d861d67d2d2441f3bd6f4eb32b10acb5/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:12.272292 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:12.279280 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e87bb0-a46c-4d3e-a95b-ee4fbc891569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55702662947b4f8ab4f9533b9d0172bc, user_id=d861d67d2d2441f3bd6f4eb32b10acb5, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:12.282714 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e87bb0-a46c-4d3e-a95b-ee4fbc891569 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:12.305479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1282/2548] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:12 2022] PUT /identity/v3/projects/55702662947b4f8ab4f9533b9d0172bc/users/d861d67d2d2441f3bd6f4eb32b10acb5/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:12.320150 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fe02f-a857-4cb5-afcb-ad7d5852f516 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:12.320423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fe02f-a857-4cb5-afcb-ad7d5852f516 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:12.320713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fe02f-a857-4cb5-afcb-ad7d5852f516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:12.353488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c32fe02f-a857-4cb5-afcb-ad7d5852f516 None None] MFA Rules not processed for user `d861d67d2d2441f3bd6f4eb32b10acb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:12.354806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c32fe02f-a857-4cb5-afcb-ad7d5852f516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:12.359329 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:12.575304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1267/2549] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:12.581412 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e87bb0-a46c-4d3e-a95b-ee4fbc891569 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:12.652171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e87bb0-a46c-4d3e-a95b-ee4fbc891569 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:12.658306 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937947e-5dfe-4c9e-bc37-5eeb3a221d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:12.659191 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937947e-5dfe-4c9e-bc37-5eeb3a221d3e admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:12.659485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937947e-5dfe-4c9e-bc37-5eeb3a221d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:12.659768 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937947e-5dfe-4c9e-bc37-5eeb3a221d3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:12.661453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937947e-5dfe-4c9e-bc37-5eeb3a221d3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:12.663811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8937947e-5dfe-4c9e-bc37-5eeb3a221d3e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:12.694229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1283/2550] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:12 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:12.698341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fe02f-a857-4cb5-afcb-ad7d5852f516 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:12.710830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fe02f-a857-4cb5-afcb-ad7d5852f516 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:12.774553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea3d3b-943f-43d6-9e47-7c7c385442c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:12.775281 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea3d3b-943f-43d6-9e47-7c7c385442c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:12.775563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea3d3b-943f-43d6-9e47-7c7c385442c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:12.775836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea3d3b-943f-43d6-9e47-7c7c385442c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:12.776571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea3d3b-943f-43d6-9e47-7c7c385442c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:12.778553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea3d3b-943f-43d6-9e47-7c7c385442c0 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:12.799909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1268/2551] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:12 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:12.804436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937947e-5dfe-4c9e-bc37-5eeb3a221d3e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:12.815310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937947e-5dfe-4c9e-bc37-5eeb3a221d3e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:12.821169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ed4cc-ebf1-4a6a-af26-cde6de33d537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:12.821873 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ed4cc-ebf1-4a6a-af26-cde6de33d537 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:12.822162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ed4cc-ebf1-4a6a-af26-cde6de33d537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:12.822446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ed4cc-ebf1-4a6a-af26-cde6de33d537 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:12.823245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ed4cc-ebf1-4a6a-af26-cde6de33d537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:12.824945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ed4cc-ebf1-4a6a-af26-cde6de33d537 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:12.837993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1284/2552] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:38:12.842739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea3d3b-943f-43d6-9e47-7c7c385442c0 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:12.853745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea3d3b-943f-43d6-9e47-7c7c385442c0 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:12.858909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee59107-1e28-4ad5-8ec7-c81a123c1bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:12.859818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee59107-1e28-4ad5-8ec7-c81a123c1bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:12.860100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee59107-1e28-4ad5-8ec7-c81a123c1bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:12.860375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee59107-1e28-4ad5-8ec7-c81a123c1bc2 admin admin] PATH_INFO: `/v3/projects/f820811ef44f4f35b38383769f29f4b3/users/c21df1cda3a6404d883f05115b971d02/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:12.883900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee59107-1e28-4ad5-8ec7-c81a123c1bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f820811ef44f4f35b38383769f29f4b3, user_id=c21df1cda3a6404d883f05115b971d02, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:12.888882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee59107-1e28-4ad5-8ec7-c81a123c1bc2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:12.907229 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1269/2553] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:12 2022] PUT /identity/v3/projects/f820811ef44f4f35b38383769f29f4b3/users/c21df1cda3a6404d883f05115b971d02/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:12.912051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ed4cc-ebf1-4a6a-af26-cde6de33d537 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:12.924605 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ed4cc-ebf1-4a6a-af26-cde6de33d537 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:12.995143 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23cd65-5abf-4a3d-9e25-2a436794fc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:12.995866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23cd65-5abf-4a3d-9e25-2a436794fc90 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:12.996072 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23cd65-5abf-4a3d-9e25-2a436794fc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:12.996981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23cd65-5abf-4a3d-9e25-2a436794fc90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:12.997735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23cd65-5abf-4a3d-9e25-2a436794fc90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:12.999400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23cd65-5abf-4a3d-9e25-2a436794fc90 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:13.011882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1285/2554] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:12 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:13.019028 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee59107-1e28-4ad5-8ec7-c81a123c1bc2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:13.032291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee59107-1e28-4ad5-8ec7-c81a123c1bc2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:13.039202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0387be1-3afc-488b-b832-949eaea90b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:13.040118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0387be1-3afc-488b-b832-949eaea90b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:13.040399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0387be1-3afc-488b-b832-949eaea90b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:13.040676 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0387be1-3afc-488b-b832-949eaea90b98 admin admin] PATH_INFO: `/v3/projects/f820811ef44f4f35b38383769f29f4b3/users/c21df1cda3a6404d883f05115b971d02/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:13.043163 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0387be1-3afc-488b-b832-949eaea90b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f820811ef44f4f35b38383769f29f4b3, user_id=c21df1cda3a6404d883f05115b971d02, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:13.045849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0387be1-3afc-488b-b832-949eaea90b98 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:13.058575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1270/2555] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:13 2022] PUT /identity/v3/projects/f820811ef44f4f35b38383769f29f4b3/users/c21df1cda3a6404d883f05115b971d02/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:13.071361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233a7c2-6338-4c86-9325-1eea69feeb3b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:13.071674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233a7c2-6338-4c86-9325-1eea69feeb3b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:13.071917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233a7c2-6338-4c86-9325-1eea69feeb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:13.105751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-1233a7c2-6338-4c86-9325-1eea69feeb3b None None] MFA Rules not processed for user `c21df1cda3a6404d883f05115b971d02`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:13.107236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1233a7c2-6338-4c86-9325-1eea69feeb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:13.162505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:13.188718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1286/2556] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:13.202763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0387be1-3afc-488b-b832-949eaea90b98 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:13.204879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f0387be1-3afc-488b-b832-949eaea90b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:13.272877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0387be1-3afc-488b-b832-949eaea90b98 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:13.276757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61425330-3f7f-41ab-a83c-57537e89f5b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:13.277409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61425330-3f7f-41ab-a83c-57537e89f5b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:13.277699 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61425330-3f7f-41ab-a83c-57537e89f5b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:13.277978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61425330-3f7f-41ab-a83c-57537e89f5b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:13.288410 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61425330-3f7f-41ab-a83c-57537e89f5b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:13.289906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61425330-3f7f-41ab-a83c-57537e89f5b9 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:13.295761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:13.302680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1271/2557] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:38:13.396583 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233a7c2-6338-4c86-9325-1eea69feeb3b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:13.407848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233a7c2-6338-4c86-9325-1eea69feeb3b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:13.412100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0a8cd-9cdf-49d9-86c4-d4b27392bd3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:13.412770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0a8cd-9cdf-49d9-86c4-d4b27392bd3b service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:13.413051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0a8cd-9cdf-49d9-86c4-d4b27392bd3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:13.413325 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0a8cd-9cdf-49d9-86c4-d4b27392bd3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:13.479544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0a8cd-9cdf-49d9-86c4-d4b27392bd3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:13.481007 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0a8cd-9cdf-49d9-86c4-d4b27392bd3b service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:13.491753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1287/2558] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:13 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:18.326258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61425330-3f7f-41ab-a83c-57537e89f5b9 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:18.398750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61425330-3f7f-41ab-a83c-57537e89f5b9 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:18.403855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:18.404734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:18.405022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:18.405302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] PATH_INFO: `/v3/users/d861d67d2d2441f3bd6f4eb32b10acb5` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:18.406557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d861d67d2d2441f3bd6f4eb32b10acb5)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:18.408542 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:18.461318 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd861d67d2d2441f3bd6f4eb32b10acb5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:18.468575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd861d67d2d2441f3bd6f4eb32b10acb5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:18.478117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd861d67d2d2441f3bd6f4eb32b10acb5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:18.502951 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd861d67d2d2441f3bd6f4eb32b10acb5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:18.503698 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd861d67d2d2441f3bd6f4eb32b10acb5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:18.511982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1272/2559] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:18 2022] DELETE /identity/v3/users/d861d67d2d2441f3bd6f4eb32b10acb5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:18.516810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5c7e3-3837-4a91-8c3f-c27eb73422a6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:18.517313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5c7e3-3837-4a91-8c3f-c27eb73422a6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:18.517456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5c7e3-3837-4a91-8c3f-c27eb73422a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:18.538962 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8ec5c7e3-3837-4a91-8c3f-c27eb73422a6 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:18.540218 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8ec5c7e3-3837-4a91-8c3f-c27eb73422a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:18.616848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1288/2560] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:18.627576 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:18.629101 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:18.702631 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f87b99-8d3e-435f-9eef-ddb4342a2e3b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:18.706500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149b5f3-1b70-43db-912f-6f4cd2caa156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:18.707209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149b5f3-1b70-43db-912f-6f4cd2caa156 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:18.707498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149b5f3-1b70-43db-912f-6f4cd2caa156 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:18.707780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149b5f3-1b70-43db-912f-6f4cd2caa156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:18.719141 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149b5f3-1b70-43db-912f-6f4cd2caa156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:18.720681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149b5f3-1b70-43db-912f-6f4cd2caa156 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:18.732007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1273/2561] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:38:19.043499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5c7e3-3837-4a91-8c3f-c27eb73422a6 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:19.045113 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8ec5c7e3-3837-4a91-8c3f-c27eb73422a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:19.062863 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5c7e3-3837-4a91-8c3f-c27eb73422a6 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:19.067744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:19.068717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:19.069003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:19.069289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] PATH_INFO: `/v3/projects/55702662947b4f8ab4f9533b9d0172bc` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:19.070526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=55702662947b4f8ab4f9533b9d0172bc)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:19.072611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:19.129394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55702662947b4f8ab4f9533b9d0172bc', 'request_id': 'req-697b04f5-32a8-4ca5-965d-6de7be387b0b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.137659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55702662947b4f8ab4f9533b9d0172bc', 'request_id': 'req-697b04f5-32a8-4ca5-965d-6de7be387b0b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.145226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55702662947b4f8ab4f9533b9d0172bc', 'request_id': 'req-697b04f5-32a8-4ca5-965d-6de7be387b0b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.156701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] The token cache is being invalidate because project 55702662947b4f8ab4f9533b9d0172bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:19.156900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.157555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.159165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1289/2562] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:19 2022] DELETE /identity/v3/projects/55702662947b4f8ab4f9533b9d0172bc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:19.164011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149b5f3-1b70-43db-912f-6f4cd2caa156 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:19.174010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149b5f3-1b70-43db-912f-6f4cd2caa156 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:19.194128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:19.237290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e381906-0bc3-4925-a2a2-993039c45070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:19.238094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e381906-0bc3-4925-a2a2-993039c45070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:19.238375 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e381906-0bc3-4925-a2a2-993039c45070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:19.238683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e381906-0bc3-4925-a2a2-993039c45070 admin admin] PATH_INFO: `/v3/users/c21df1cda3a6404d883f05115b971d02` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:19.240295 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e381906-0bc3-4925-a2a2-993039c45070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c21df1cda3a6404d883f05115b971d02)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:19.242250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e381906-0bc3-4925-a2a2-993039c45070 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:19.296921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3e381906-0bc3-4925-a2a2-993039c45070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c21df1cda3a6404d883f05115b971d02'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.305580 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3e381906-0bc3-4925-a2a2-993039c45070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c21df1cda3a6404d883f05115b971d02'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.314025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3e381906-0bc3-4925-a2a2-993039c45070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c21df1cda3a6404d883f05115b971d02'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.329026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3e381906-0bc3-4925-a2a2-993039c45070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c21df1cda3a6404d883f05115b971d02'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.329820 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-3e381906-0bc3-4925-a2a2-993039c45070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c21df1cda3a6404d883f05115b971d02'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.337333 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1274/2563] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:19 2022] DELETE /identity/v3/users/c21df1cda3a6404d883f05115b971d02 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:19.685048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:19.686485 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:19.705709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b04f5-32a8-4ca5-965d-6de7be387b0b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:19.712244 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:19.786296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:19.787912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:19.788311 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:19.788594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] PATH_INFO: `/v3/projects/f820811ef44f4f35b38383769f29f4b3` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:19.789876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f820811ef44f4f35b38383769f29f4b3)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:19.791933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:19.855605 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f820811ef44f4f35b38383769f29f4b3', 'request_id': 'req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.864068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f820811ef44f4f35b38383769f29f4b3', 'request_id': 'req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.876560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f820811ef44f4f35b38383769f29f4b3', 'request_id': 'req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.891845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] The token cache is being invalidate because project f820811ef44f4f35b38383769f29f4b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:19.892319 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.892807 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-572beb8e-ed50-4cd7-8e0f-7fb8b02bd43e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:19.894917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1290/2564] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:19 2022] DELETE /identity/v3/projects/f820811ef44f4f35b38383769f29f4b3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:38:19.900940 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876da0d0-ccf5-477e-a638-5aa824b9a02c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:19.901230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876da0d0-ccf5-477e-a638-5aa824b9a02c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:19.901511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876da0d0-ccf5-477e-a638-5aa824b9a02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:19.924396 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-876da0d0-ccf5-477e-a638-5aa824b9a02c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:19.925638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-876da0d0-ccf5-477e-a638-5aa824b9a02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:20.000147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1275/2565] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:38:20.012526 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d35375-34d3-414a-936f-ffd8175bc97a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.012893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d35375-34d3-414a-936f-ffd8175bc97a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.013293 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d35375-34d3-414a-936f-ffd8175bc97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.037769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f1d35375-34d3-414a-936f-ffd8175bc97a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:20.039227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f1d35375-34d3-414a-936f-ffd8175bc97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:20.049579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1291/2566] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:20.055435 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876da0d0-ccf5-477e-a638-5aa824b9a02c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:20.076384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876da0d0-ccf5-477e-a638-5aa824b9a02c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:20.084372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ea0f9-f9e0-4c8f-bacf-1a0b801efe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:20.085534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea0f9-f9e0-4c8f-bacf-1a0b801efe8b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.086050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea0f9-f9e0-4c8f-bacf-1a0b801efe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.086555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ea0f9-f9e0-4c8f-bacf-1a0b801efe8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.087997 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ea0f9-f9e0-4c8f-bacf-1a0b801efe8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:38:20.088573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ea0f9-f9e0-4c8f-bacf-1a0b801efe8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:20.091600 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ea0f9-f9e0-4c8f-bacf-1a0b801efe8b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:20.113417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1276/2567] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:38:20.118450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d35375-34d3-414a-936f-ffd8175bc97a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:20.129552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d35375-34d3-414a-936f-ffd8175bc97a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:20.136609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bde95-daec-4c9e-8222-447969dae327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:20.137309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bde95-daec-4c9e-8222-447969dae327 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.137601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bde95-daec-4c9e-8222-447969dae327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.137871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bde95-daec-4c9e-8222-447969dae327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.139499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bde95-daec-4c9e-8222-447969dae327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:20.141535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bde95-daec-4c9e-8222-447969dae327 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:20.179768 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1292/2568] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:20 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:20.186835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ea0f9-f9e0-4c8f-bacf-1a0b801efe8b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:20.199294 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ea0f9-f9e0-4c8f-bacf-1a0b801efe8b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:20.233431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:20.278467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36850bb3-799e-4a0a-accf-353c960912d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:20.279232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36850bb3-799e-4a0a-accf-353c960912d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.279651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36850bb3-799e-4a0a-accf-353c960912d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.282124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36850bb3-799e-4a0a-accf-353c960912d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.282124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36850bb3-799e-4a0a-accf-353c960912d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:20.284919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36850bb3-799e-4a0a-accf-353c960912d7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:20.309573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1277/2569] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:20 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:20.314034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bde95-daec-4c9e-8222-447969dae327 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:20.327484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bde95-daec-4c9e-8222-447969dae327 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:20.336239 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627145c-1e72-4613-807c-7d0aa254210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:20.336998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627145c-1e72-4613-807c-7d0aa254210b admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.337385 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627145c-1e72-4613-807c-7d0aa254210b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.337855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627145c-1e72-4613-807c-7d0aa254210b admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.338796 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627145c-1e72-4613-807c-7d0aa254210b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:20.340610 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627145c-1e72-4613-807c-7d0aa254210b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:20.357533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1293/2570] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:20.362993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36850bb3-799e-4a0a-accf-353c960912d7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:20.374683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36850bb3-799e-4a0a-accf-353c960912d7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:20.379834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b5c962-5a9d-4783-9646-c43fe4e1689b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:20.380559 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5c962-5a9d-4783-9646-c43fe4e1689b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.380873 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5c962-5a9d-4783-9646-c43fe4e1689b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.381289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b5c962-5a9d-4783-9646-c43fe4e1689b admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.381888 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5c962-5a9d-4783-9646-c43fe4e1689b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:20.383700 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b5c962-5a9d-4783-9646-c43fe4e1689b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:20.396770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1278/2571] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:20.401650 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627145c-1e72-4613-807c-7d0aa254210b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:20.413254 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627145c-1e72-4613-807c-7d0aa254210b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:20.418544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3b706-2eb7-416c-9c7d-0abe2cdde9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:20.419535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3b706-2eb7-416c-9c7d-0abe2cdde9ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.419842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3b706-2eb7-416c-9c7d-0abe2cdde9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.420125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3b706-2eb7-416c-9c7d-0abe2cdde9ce admin admin] PATH_INFO: `/v3/projects/960e2dcc63f64816a9f3b0bf711972be/users/4f1f01aec2bc4c1588bfaf8143650f83/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.439534 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3b706-2eb7-416c-9c7d-0abe2cdde9ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=960e2dcc63f64816a9f3b0bf711972be, user_id=4f1f01aec2bc4c1588bfaf8143650f83, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:20.442620 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3b706-2eb7-416c-9c7d-0abe2cdde9ce admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:20.459127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1294/2572] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:20 2022] PUT /identity/v3/projects/960e2dcc63f64816a9f3b0bf711972be/users/4f1f01aec2bc4c1588bfaf8143650f83/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:20.470921 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f7fbc-54e6-45b8-82a2-6988b168a851 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.471240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f7fbc-54e6-45b8-82a2-6988b168a851 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.471526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f7fbc-54e6-45b8-82a2-6988b168a851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.502328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a02f7fbc-54e6-45b8-82a2-6988b168a851 None None] MFA Rules not processed for user `4f1f01aec2bc4c1588bfaf8143650f83`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:20.503618 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a02f7fbc-54e6-45b8-82a2-6988b168a851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:20.576117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1279/2573] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:38:20.581592 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3b706-2eb7-416c-9c7d-0abe2cdde9ce admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:20.619197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:20.653892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3b706-2eb7-416c-9c7d-0abe2cdde9ce admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:20.659162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa3eb4-d4f9-44fa-b9f3-ba5e9370478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:20.659841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa3eb4-d4f9-44fa-b9f3-ba5e9370478a admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.660437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa3eb4-d4f9-44fa-b9f3-ba5e9370478a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.660600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa3eb4-d4f9-44fa-b9f3-ba5e9370478a admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.661318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa3eb4-d4f9-44fa-b9f3-ba5e9370478a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:20.664248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa3eb4-d4f9-44fa-b9f3-ba5e9370478a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:20.699114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1295/2574] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:20 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:20.701594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f7fbc-54e6-45b8-82a2-6988b168a851 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:20.713840 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f7fbc-54e6-45b8-82a2-6988b168a851 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:20.782021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62aeafb-6d3a-4365-ade6-aff59a28eb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:20.782736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62aeafb-6d3a-4365-ade6-aff59a28eb17 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.783022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62aeafb-6d3a-4365-ade6-aff59a28eb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.783300 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62aeafb-6d3a-4365-ade6-aff59a28eb17 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.784010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62aeafb-6d3a-4365-ade6-aff59a28eb17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:20.785942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62aeafb-6d3a-4365-ade6-aff59a28eb17 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:20.809638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1280/2575] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:20 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:20.814659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa3eb4-d4f9-44fa-b9f3-ba5e9370478a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:20.825613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa3eb4-d4f9-44fa-b9f3-ba5e9370478a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:20.830689 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca51770-b115-454d-a6fa-2c370a0d69de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:20.831308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca51770-b115-454d-a6fa-2c370a0d69de admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.831594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca51770-b115-454d-a6fa-2c370a0d69de admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.831877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca51770-b115-454d-a6fa-2c370a0d69de admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.832618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca51770-b115-454d-a6fa-2c370a0d69de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:20.834781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca51770-b115-454d-a6fa-2c370a0d69de admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:20.847669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1296/2576] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:20.852879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62aeafb-6d3a-4365-ade6-aff59a28eb17 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:20.865412 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62aeafb-6d3a-4365-ade6-aff59a28eb17 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:20.872919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757610f-4465-4fdb-af91-9424a8571a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:20.873836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757610f-4465-4fdb-af91-9424a8571a14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:20.874123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757610f-4465-4fdb-af91-9424a8571a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:20.874407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757610f-4465-4fdb-af91-9424a8571a14 admin admin] PATH_INFO: `/v3/projects/da6bf39ec4c64c30a4e5aac2b99bbac1/users/61b6211cfa08471d92ebf0e908c955ae/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:20.891642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757610f-4465-4fdb-af91-9424a8571a14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6bf39ec4c64c30a4e5aac2b99bbac1, user_id=61b6211cfa08471d92ebf0e908c955ae, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:20.894337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757610f-4465-4fdb-af91-9424a8571a14 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:20.910298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1281/2577] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:20 2022] PUT /identity/v3/projects/da6bf39ec4c64c30a4e5aac2b99bbac1/users/61b6211cfa08471d92ebf0e908c955ae/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:20.915731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca51770-b115-454d-a6fa-2c370a0d69de admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:20.926505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca51770-b115-454d-a6fa-2c370a0d69de admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:20.999040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcd26e-2700-4d90-8d07-f33ac73d4826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:20.999725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcd26e-2700-4d90-8d07-f33ac73d4826 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:21.000038 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcd26e-2700-4d90-8d07-f33ac73d4826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:21.000308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcd26e-2700-4d90-8d07-f33ac73d4826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:21.001043 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcd26e-2700-4d90-8d07-f33ac73d4826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:21.002710 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcd26e-2700-4d90-8d07-f33ac73d4826 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:21.015311 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1297/2578] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:38:21.023234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757610f-4465-4fdb-af91-9424a8571a14 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:21.033315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:21.043233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757610f-4465-4fdb-af91-9424a8571a14 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:21.049734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546f930-b570-4990-be21-f3fd3e0d1e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:21.050670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546f930-b570-4990-be21-f3fd3e0d1e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:21.050963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546f930-b570-4990-be21-f3fd3e0d1e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:21.051240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546f930-b570-4990-be21-f3fd3e0d1e32 admin admin] PATH_INFO: `/v3/projects/da6bf39ec4c64c30a4e5aac2b99bbac1/users/61b6211cfa08471d92ebf0e908c955ae/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:21.053979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546f930-b570-4990-be21-f3fd3e0d1e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da6bf39ec4c64c30a4e5aac2b99bbac1, user_id=61b6211cfa08471d92ebf0e908c955ae, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:21.056675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546f930-b570-4990-be21-f3fd3e0d1e32 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:21.067636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1282/2579] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:21 2022] PUT /identity/v3/projects/da6bf39ec4c64c30a4e5aac2b99bbac1/users/61b6211cfa08471d92ebf0e908c955ae/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:21.078941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b313e-bb93-44ff-ae81-4be0b8edbb92 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:21.079234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b313e-bb93-44ff-ae81-4be0b8edbb92 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:21.079518 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b313e-bb93-44ff-ae81-4be0b8edbb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:21.109676 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-707b313e-bb93-44ff-ae81-4be0b8edbb92 None None] MFA Rules not processed for user `61b6211cfa08471d92ebf0e908c955ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:21.110941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-707b313e-bb93-44ff-ae81-4be0b8edbb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:21.189186 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1298/2580] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:21.212571 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546f930-b570-4990-be21-f3fd3e0d1e32 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:21.214472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c546f930-b570-4990-be21-f3fd3e0d1e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:21.277335 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546f930-b570-4990-be21-f3fd3e0d1e32 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:21.286345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a4ca0-5321-422d-9d92-77e74248672d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:21.287607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a4ca0-5321-422d-9d92-77e74248672d service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:21.287927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a4ca0-5321-422d-9d92-77e74248672d service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:21.288210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a4ca0-5321-422d-9d92-77e74248672d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:21.298155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a4ca0-5321-422d-9d92-77e74248672d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:21.299664 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a4ca0-5321-422d-9d92-77e74248672d service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:21.311205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1283/2581] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:38:21.393313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b313e-bb93-44ff-ae81-4be0b8edbb92 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:21.404043 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b313e-bb93-44ff-ae81-4be0b8edbb92 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:21.407772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f332b-3bc2-4ba7-a9b9-4a64d9e6fa4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:21.408405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f332b-3bc2-4ba7-a9b9-4a64d9e6fa4d service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:21.408691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f332b-3bc2-4ba7-a9b9-4a64d9e6fa4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:21.408995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f332b-3bc2-4ba7-a9b9-4a64d9e6fa4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:21.436078 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:21.470436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f332b-3bc2-4ba7-a9b9-4a64d9e6fa4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:21.475034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f332b-3bc2-4ba7-a9b9-4a64d9e6fa4d service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:21.483507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1299/2582] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:21.748671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a4ca0-5321-422d-9d92-77e74248672d service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:21.831602 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a4ca0-5321-422d-9d92-77e74248672d service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:21.837280 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:21.838186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:21.838473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:21.838789 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] PATH_INFO: `/v3/users/4f1f01aec2bc4c1588bfaf8143650f83` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:21.840032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f1f01aec2bc4c1588bfaf8143650f83)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:21.841991 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:21.894278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f1f01aec2bc4c1588bfaf8143650f83'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:21.902920 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f1f01aec2bc4c1588bfaf8143650f83'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:21.912253 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f1f01aec2bc4c1588bfaf8143650f83'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:21.925866 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f1f01aec2bc4c1588bfaf8143650f83'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:21.926630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f1f01aec2bc4c1588bfaf8143650f83'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:21.932913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1284/2583] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:21 2022] DELETE /identity/v3/users/4f1f01aec2bc4c1588bfaf8143650f83 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:21.938516 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a6424-5d89-4451-af79-c897b126d2d6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:21.939095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a6424-5d89-4451-af79-c897b126d2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:21.939095 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a6424-5d89-4451-af79-c897b126d2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:21.956966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-103a6424-5d89-4451-af79-c897b126d2d6 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:21.958206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-103a6424-5d89-4451-af79-c897b126d2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:22.027827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1300/2584] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:22.038090 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:22.039585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:22.054507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:22.102934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f95b05-cc1e-4008-a456-283731a8f78c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:22.106626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac91d32-5563-4834-b2f0-f65a4dd1e36c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:22.107291 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac91d32-5563-4834-b2f0-f65a4dd1e36c service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:22.107581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac91d32-5563-4834-b2f0-f65a4dd1e36c service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:22.107863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac91d32-5563-4834-b2f0-f65a4dd1e36c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:22.118269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac91d32-5563-4834-b2f0-f65a4dd1e36c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:22.119779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac91d32-5563-4834-b2f0-f65a4dd1e36c service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:22.129573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1285/2585] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:22.450614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a6424-5d89-4451-af79-c897b126d2d6 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:22.452495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-103a6424-5d89-4451-af79-c897b126d2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:22.467558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a6424-5d89-4451-af79-c897b126d2d6 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:22.472705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:22.473587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:22.473876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:22.474195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] PATH_INFO: `/v3/projects/960e2dcc63f64816a9f3b0bf711972be` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:22.475704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=960e2dcc63f64816a9f3b0bf711972be)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:22.478480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:22.541366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '960e2dcc63f64816a9f3b0bf711972be', 'request_id': 'req-b574e996-704d-4573-a50d-84ad3eb3ee32'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:22.550099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '960e2dcc63f64816a9f3b0bf711972be', 'request_id': 'req-b574e996-704d-4573-a50d-84ad3eb3ee32'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:22.557746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '960e2dcc63f64816a9f3b0bf711972be', 'request_id': 'req-b574e996-704d-4573-a50d-84ad3eb3ee32'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:22.568712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] The token cache is being invalidate because project 960e2dcc63f64816a9f3b0bf711972be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:22.569521 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:22.570194 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:22.571900 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1301/2586] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:22 2022] DELETE /identity/v3/projects/960e2dcc63f64816a9f3b0bf711972be => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:22.576058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac91d32-5563-4834-b2f0-f65a4dd1e36c service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:22.585960 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac91d32-5563-4834-b2f0-f65a4dd1e36c service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:22.648328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd998c-34fc-4c50-be01-febbeae52f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:22.649202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd998c-34fc-4c50-be01-febbeae52f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:22.649487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd998c-34fc-4c50-be01-febbeae52f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:22.649764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd998c-34fc-4c50-be01-febbeae52f99 admin admin] PATH_INFO: `/v3/users/61b6211cfa08471d92ebf0e908c955ae` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:22.651002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd998c-34fc-4c50-be01-febbeae52f99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61b6211cfa08471d92ebf0e908c955ae)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:22.652935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd998c-34fc-4c50-be01-febbeae52f99 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:22.709048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-30bd998c-34fc-4c50-be01-febbeae52f99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61b6211cfa08471d92ebf0e908c955ae'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:22.717605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-30bd998c-34fc-4c50-be01-febbeae52f99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61b6211cfa08471d92ebf0e908c955ae'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:22.726126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-30bd998c-34fc-4c50-be01-febbeae52f99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61b6211cfa08471d92ebf0e908c955ae'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:22.740559 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-30bd998c-34fc-4c50-be01-febbeae52f99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61b6211cfa08471d92ebf0e908c955ae'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:22.741362 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-30bd998c-34fc-4c50-be01-febbeae52f99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61b6211cfa08471d92ebf0e908c955ae'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:22.748245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1286/2587] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:22 2022] DELETE /identity/v3/users/61b6211cfa08471d92ebf0e908c955ae => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:23.048575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:23.051274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:23.069682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574e996-704d-4573-a50d-84ad3eb3ee32 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:23.128137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:23.143099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:23.144184 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:23.144575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:23.144957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] PATH_INFO: `/v3/projects/da6bf39ec4c64c30a4e5aac2b99bbac1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:23.147164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da6bf39ec4c64c30a4e5aac2b99bbac1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:23.149478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:23.213441 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da6bf39ec4c64c30a4e5aac2b99bbac1', 'request_id': 'req-5db9b2c0-f7a5-4367-9ae5-e24d91934919'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:23.224295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da6bf39ec4c64c30a4e5aac2b99bbac1', 'request_id': 'req-5db9b2c0-f7a5-4367-9ae5-e24d91934919'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:23.231567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da6bf39ec4c64c30a4e5aac2b99bbac1', 'request_id': 'req-5db9b2c0-f7a5-4367-9ae5-e24d91934919'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:23.243313 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] The token cache is being invalidate because project da6bf39ec4c64c30a4e5aac2b99bbac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:23.243595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:23.244260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5db9b2c0-f7a5-4367-9ae5-e24d91934919 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:23.247320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1302/2588] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:23 2022] DELETE /identity/v3/projects/da6bf39ec4c64c30a4e5aac2b99bbac1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:23.256233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073b1b7-5f84-487e-9077-70833fbd0b38 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:23.257077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073b1b7-5f84-487e-9077-70833fbd0b38 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:23.257077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073b1b7-5f84-487e-9077-70833fbd0b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:23.277332 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e073b1b7-5f84-487e-9077-70833fbd0b38 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:23.278592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e073b1b7-5f84-487e-9077-70833fbd0b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:23.311087 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:23.348371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1287/2589] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:23.356665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41616c7a-e9df-4992-a04b-1b9e352d07ae None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:23.356963 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41616c7a-e9df-4992-a04b-1b9e352d07ae None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:23.357246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41616c7a-e9df-4992-a04b-1b9e352d07ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:23.377317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-41616c7a-e9df-4992-a04b-1b9e352d07ae None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:23.378595 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-41616c7a-e9df-4992-a04b-1b9e352d07ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:23.387511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1303/2590] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:23.392113 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073b1b7-5f84-487e-9077-70833fbd0b38 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:23.402601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e073b1b7-5f84-487e-9077-70833fbd0b38 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:23.407447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455fe37-2bc9-48e4-becb-d5740e489149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:23.408556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455fe37-2bc9-48e4-becb-d5740e489149 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:23.408846 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455fe37-2bc9-48e4-becb-d5740e489149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:23.409252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455fe37-2bc9-48e4-becb-d5740e489149 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:23.410342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455fe37-2bc9-48e4-becb-d5740e489149 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:38:23.410708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455fe37-2bc9-48e4-becb-d5740e489149 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:23.412544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b455fe37-2bc9-48e4-becb-d5740e489149 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:23.425479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1288/2591] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:38:23.431156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41616c7a-e9df-4992-a04b-1b9e352d07ae None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:23.441048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41616c7a-e9df-4992-a04b-1b9e352d07ae None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:23.446665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772ddea-16ab-4f15-8604-93740119f75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:23.447339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772ddea-16ab-4f15-8604-93740119f75f admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:23.447624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772ddea-16ab-4f15-8604-93740119f75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:23.447902 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d772ddea-16ab-4f15-8604-93740119f75f admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:23.448616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772ddea-16ab-4f15-8604-93740119f75f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:23.450603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d772ddea-16ab-4f15-8604-93740119f75f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:23.480615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1304/2592] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:23 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:23.487001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455fe37-2bc9-48e4-becb-d5740e489149 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:23.498747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455fe37-2bc9-48e4-becb-d5740e489149 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:23.561383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221296f-57a5-437f-b6a6-b0144ee962de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:23.562082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221296f-57a5-437f-b6a6-b0144ee962de admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:23.562355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221296f-57a5-437f-b6a6-b0144ee962de admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:23.562698 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f221296f-57a5-437f-b6a6-b0144ee962de admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:23.563406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221296f-57a5-437f-b6a6-b0144ee962de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:23.565405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f221296f-57a5-437f-b6a6-b0144ee962de admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:23.588432 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1289/2593] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:23 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:38:23.592985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772ddea-16ab-4f15-8604-93740119f75f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:23.604356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d772ddea-16ab-4f15-8604-93740119f75f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:23.609635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eea2e0-a65c-4b4a-b806-b012f062e695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:23.610340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eea2e0-a65c-4b4a-b806-b012f062e695 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:23.610857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eea2e0-a65c-4b4a-b806-b012f062e695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:23.611252 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eea2e0-a65c-4b4a-b806-b012f062e695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:23.612592 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eea2e0-a65c-4b4a-b806-b012f062e695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:23.614786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eea2e0-a65c-4b4a-b806-b012f062e695 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:23.630070 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1305/2594] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:38:23.635509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221296f-57a5-437f-b6a6-b0144ee962de admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:23.647352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f221296f-57a5-437f-b6a6-b0144ee962de admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:23.652133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74177724-879a-48f2-a7eb-c41800178684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:23.652805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74177724-879a-48f2-a7eb-c41800178684 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:23.653095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74177724-879a-48f2-a7eb-c41800178684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:23.653392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74177724-879a-48f2-a7eb-c41800178684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:23.654210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74177724-879a-48f2-a7eb-c41800178684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:23.655938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74177724-879a-48f2-a7eb-c41800178684 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:23.667804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1290/2595] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:23.673878 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eea2e0-a65c-4b4a-b806-b012f062e695 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:23.684695 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eea2e0-a65c-4b4a-b806-b012f062e695 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:23.689465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d7d25-d774-4b6f-8f3b-60bb0027c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:23.690361 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d7d25-d774-4b6f-8f3b-60bb0027c3bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:23.690681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d7d25-d774-4b6f-8f3b-60bb0027c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:23.690971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d7d25-d774-4b6f-8f3b-60bb0027c3bc admin admin] PATH_INFO: `/v3/projects/5b2706ac367a4b3eaf591fc6cdc4b74e/users/a321b81a2d734f8d826df217c8aebef0/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:23.707557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d7d25-d774-4b6f-8f3b-60bb0027c3bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b2706ac367a4b3eaf591fc6cdc4b74e, user_id=a321b81a2d734f8d826df217c8aebef0, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:23.711218 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d7d25-d774-4b6f-8f3b-60bb0027c3bc admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:23.726053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1306/2596] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:23 2022] PUT /identity/v3/projects/5b2706ac367a4b3eaf591fc6cdc4b74e/users/a321b81a2d734f8d826df217c8aebef0/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:23.737567 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049015b6-bd03-499d-8dc3-95808cab5c0b None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:23.737798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049015b6-bd03-499d-8dc3-95808cab5c0b None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:23.738102 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049015b6-bd03-499d-8dc3-95808cab5c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:23.769256 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-049015b6-bd03-499d-8dc3-95808cab5c0b None None] MFA Rules not processed for user `a321b81a2d734f8d826df217c8aebef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:23.770499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-049015b6-bd03-499d-8dc3-95808cab5c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:23.841566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1291/2597] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:23.846160 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d7d25-d774-4b6f-8f3b-60bb0027c3bc admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:23.914223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d7d25-d774-4b6f-8f3b-60bb0027c3bc admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:23.919285 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e5a61-5a10-4e38-88a9-02aae3c51756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:23.919958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e5a61-5a10-4e38-88a9-02aae3c51756 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:23.920246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e5a61-5a10-4e38-88a9-02aae3c51756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:23.920525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350e5a61-5a10-4e38-88a9-02aae3c51756 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:23.921419 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e5a61-5a10-4e38-88a9-02aae3c51756 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:23.923431 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350e5a61-5a10-4e38-88a9-02aae3c51756 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:23.953474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1307/2598] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:23 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:23.957296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049015b6-bd03-499d-8dc3-95808cab5c0b None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:23.968477 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049015b6-bd03-499d-8dc3-95808cab5c0b None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:23.997222 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:24.039202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc93c0-53ca-4978-a162-e4d2b47d1e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:24.039885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc93c0-53ca-4978-a162-e4d2b47d1e54 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:24.040169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc93c0-53ca-4978-a162-e4d2b47d1e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:24.040441 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc93c0-53ca-4978-a162-e4d2b47d1e54 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:24.041130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc93c0-53ca-4978-a162-e4d2b47d1e54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:24.043198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dc93c0-53ca-4978-a162-e4d2b47d1e54 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:24.063771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1292/2599] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:23 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:24.069192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e5a61-5a10-4e38-88a9-02aae3c51756 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:24.075082 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:24.081803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350e5a61-5a10-4e38-88a9-02aae3c51756 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:24.088960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06253981-3e17-44db-bdac-1b96908e5d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:24.089648 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253981-3e17-44db-bdac-1b96908e5d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:24.089936 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253981-3e17-44db-bdac-1b96908e5d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:24.090230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06253981-3e17-44db-bdac-1b96908e5d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:24.091064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06253981-3e17-44db-bdac-1b96908e5d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:24.092693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06253981-3e17-44db-bdac-1b96908e5d2f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:24.104440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1308/2600] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:24.110907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc93c0-53ca-4978-a162-e4d2b47d1e54 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:24.127244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dc93c0-53ca-4978-a162-e4d2b47d1e54 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:24.132964 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc79c7-8f31-447d-82e8-1ff1e45f0650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:24.134063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc79c7-8f31-447d-82e8-1ff1e45f0650 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:24.134226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc79c7-8f31-447d-82e8-1ff1e45f0650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:24.134417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc79c7-8f31-447d-82e8-1ff1e45f0650 admin admin] PATH_INFO: `/v3/projects/064ed24d1ffb41e68e6d67f0af0b7413/users/94f9283575ad4d7caf9aa95ffbbfb736/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:24.150616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc79c7-8f31-447d-82e8-1ff1e45f0650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064ed24d1ffb41e68e6d67f0af0b7413, user_id=94f9283575ad4d7caf9aa95ffbbfb736, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:24.153298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc79c7-8f31-447d-82e8-1ff1e45f0650 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:24.171734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1293/2601] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:24 2022] PUT /identity/v3/projects/064ed24d1ffb41e68e6d67f0af0b7413/users/94f9283575ad4d7caf9aa95ffbbfb736/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:24.176127 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06253981-3e17-44db-bdac-1b96908e5d2f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:24.187550 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06253981-3e17-44db-bdac-1b96908e5d2f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:24.249538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419f44bf-ccb8-4ad7-9d4f-24f9adad9f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:24.250074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f44bf-ccb8-4ad7-9d4f-24f9adad9f28 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:24.250375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f44bf-ccb8-4ad7-9d4f-24f9adad9f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:24.250672 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f44bf-ccb8-4ad7-9d4f-24f9adad9f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:24.251467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f44bf-ccb8-4ad7-9d4f-24f9adad9f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:24.253246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419f44bf-ccb8-4ad7-9d4f-24f9adad9f28 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:24.271719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1309/2602] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:24 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:24.276839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc79c7-8f31-447d-82e8-1ff1e45f0650 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:24.288640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc79c7-8f31-447d-82e8-1ff1e45f0650 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:24.293585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78438b9b-0eb0-4dd3-b163-98c9765b5f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:24.294470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78438b9b-0eb0-4dd3-b163-98c9765b5f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:24.294784 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78438b9b-0eb0-4dd3-b163-98c9765b5f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:24.295065 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78438b9b-0eb0-4dd3-b163-98c9765b5f36 admin admin] PATH_INFO: `/v3/projects/064ed24d1ffb41e68e6d67f0af0b7413/users/94f9283575ad4d7caf9aa95ffbbfb736/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:24.297085 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78438b9b-0eb0-4dd3-b163-98c9765b5f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064ed24d1ffb41e68e6d67f0af0b7413, user_id=94f9283575ad4d7caf9aa95ffbbfb736, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:24.299776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78438b9b-0eb0-4dd3-b163-98c9765b5f36 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:24.312047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1294/2603] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:24 2022] PUT /identity/v3/projects/064ed24d1ffb41e68e6d67f0af0b7413/users/94f9283575ad4d7caf9aa95ffbbfb736/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:24.327713 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d366450-7c8d-407a-a491-4949ec6822ac None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:24.328006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d366450-7c8d-407a-a491-4949ec6822ac None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:24.328286 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d366450-7c8d-407a-a491-4949ec6822ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:24.357328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7d366450-7c8d-407a-a491-4949ec6822ac None None] MFA Rules not processed for user `94f9283575ad4d7caf9aa95ffbbfb736`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:24.358564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7d366450-7c8d-407a-a491-4949ec6822ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:24.430539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1310/2604] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:24.442077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78438b9b-0eb0-4dd3-b163-98c9765b5f36 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:24.445319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-78438b9b-0eb0-4dd3-b163-98c9765b5f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:24.507724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78438b9b-0eb0-4dd3-b163-98c9765b5f36 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:24.511272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11559f4f-a018-4c06-b453-9c9eb21b3296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:24.511924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11559f4f-a018-4c06-b453-9c9eb21b3296 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:24.512215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11559f4f-a018-4c06-b453-9c9eb21b3296 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:24.512491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11559f4f-a018-4c06-b453-9c9eb21b3296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:24.572906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11559f4f-a018-4c06-b453-9c9eb21b3296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:24.574358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11559f4f-a018-4c06-b453-9c9eb21b3296 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:24.583870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1295/2605] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:27.134612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d366450-7c8d-407a-a491-4949ec6822ac None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:27.201627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d366450-7c8d-407a-a491-4949ec6822ac None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:27.206168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:27.207092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:27.207375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:27.207647 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] PATH_INFO: `/v3/users/a321b81a2d734f8d826df217c8aebef0` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:27.208848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a321b81a2d734f8d826df217c8aebef0)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:27.210876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:27.225626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:27.262609 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a321b81a2d734f8d826df217c8aebef0'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:27.275177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a321b81a2d734f8d826df217c8aebef0'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:27.275827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a321b81a2d734f8d826df217c8aebef0'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:27.284409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a321b81a2d734f8d826df217c8aebef0'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:27.292521 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a321b81a2d734f8d826df217c8aebef0'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:27.300197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1311/2606] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:27 2022] DELETE /identity/v3/users/a321b81a2d734f8d826df217c8aebef0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:27.305226 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94b098-3965-4efa-998e-70201b1c82f5 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:27.305746 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94b098-3965-4efa-998e-70201b1c82f5 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:27.305992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94b098-3965-4efa-998e-70201b1c82f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:27.324250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-de94b098-3965-4efa-998e-70201b1c82f5 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:27.325458 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-de94b098-3965-4efa-998e-70201b1c82f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:27.357994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:27.390407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1296/2607] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:27.400896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:27.402423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:27.463514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46754c7d-1814-43a5-9bc9-28119a9fa302 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:27.467275 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f74f53-cba8-4391-ae7f-f6dd0e2f425b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:27.467922 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74f53-cba8-4391-ae7f-f6dd0e2f425b service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:27.468206 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74f53-cba8-4391-ae7f-f6dd0e2f425b service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:27.468501 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f74f53-cba8-4391-ae7f-f6dd0e2f425b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:27.479026 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f74f53-cba8-4391-ae7f-f6dd0e2f425b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:27.480567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f74f53-cba8-4391-ae7f-f6dd0e2f425b service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:27.490600 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1312/2608] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:27.792993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94b098-3965-4efa-998e-70201b1c82f5 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:27.794553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-de94b098-3965-4efa-998e-70201b1c82f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:27.812541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94b098-3965-4efa-998e-70201b1c82f5 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:27.817615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:27.818448 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:27.818714 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:27.819096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] PATH_INFO: `/v3/projects/5b2706ac367a4b3eaf591fc6cdc4b74e` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:27.820392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b2706ac367a4b3eaf591fc6cdc4b74e)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:27.823394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:27.885118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b2706ac367a4b3eaf591fc6cdc4b74e', 'request_id': 'req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:27.891713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b2706ac367a4b3eaf591fc6cdc4b74e', 'request_id': 'req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:27.899817 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b2706ac367a4b3eaf591fc6cdc4b74e', 'request_id': 'req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:27.913907 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] The token cache is being invalidate because project 5b2706ac367a4b3eaf591fc6cdc4b74e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:27.914206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:27.915072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:27.916770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1297/2609] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:27 2022] DELETE /identity/v3/projects/5b2706ac367a4b3eaf591fc6cdc4b74e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:27.922511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f74f53-cba8-4391-ae7f-f6dd0e2f425b service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:27.932913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f74f53-cba8-4391-ae7f-f6dd0e2f425b service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:27.998083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886dc6b-9b51-491e-8cda-972caa6c0d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:27.999046 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886dc6b-9b51-491e-8cda-972caa6c0d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:27.999331 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886dc6b-9b51-491e-8cda-972caa6c0d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:27.999632 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886dc6b-9b51-491e-8cda-972caa6c0d97 admin admin] PATH_INFO: `/v3/users/94f9283575ad4d7caf9aa95ffbbfb736` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:28.001307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886dc6b-9b51-491e-8cda-972caa6c0d97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94f9283575ad4d7caf9aa95ffbbfb736)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:28.003357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886dc6b-9b51-491e-8cda-972caa6c0d97 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:28.054187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:28.066950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e886dc6b-9b51-491e-8cda-972caa6c0d97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94f9283575ad4d7caf9aa95ffbbfb736'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:28.082028 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e886dc6b-9b51-491e-8cda-972caa6c0d97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94f9283575ad4d7caf9aa95ffbbfb736'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:28.082951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e886dc6b-9b51-491e-8cda-972caa6c0d97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94f9283575ad4d7caf9aa95ffbbfb736'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:28.091739 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e886dc6b-9b51-491e-8cda-972caa6c0d97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94f9283575ad4d7caf9aa95ffbbfb736'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:28.100214 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e886dc6b-9b51-491e-8cda-972caa6c0d97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94f9283575ad4d7caf9aa95ffbbfb736'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:28.106957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1313/2610] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:27 2022] DELETE /identity/v3/users/94f9283575ad4d7caf9aa95ffbbfb736 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:28.420882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:28.422369 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:28.438953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1fa3f-6302-49f5-97f0-3ae7ae63f27e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:28.509594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:28.510462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:28.510770 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:28.511042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] PATH_INFO: `/v3/projects/064ed24d1ffb41e68e6d67f0af0b7413` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:28.512258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=064ed24d1ffb41e68e6d67f0af0b7413)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:28.514168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:28.565712 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064ed24d1ffb41e68e6d67f0af0b7413', 'request_id': 'req-b262d8d4-90ab-4d67-a0da-186d71e012e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:28.571440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064ed24d1ffb41e68e6d67f0af0b7413', 'request_id': 'req-b262d8d4-90ab-4d67-a0da-186d71e012e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:28.573367 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:28.578139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064ed24d1ffb41e68e6d67f0af0b7413', 'request_id': 'req-b262d8d4-90ab-4d67-a0da-186d71e012e9'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:28.594561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] The token cache is being invalidate because project 064ed24d1ffb41e68e6d67f0af0b7413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:28.594867 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:28.595686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b262d8d4-90ab-4d67-a0da-186d71e012e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:28.597365 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1298/2611] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:28 2022] DELETE /identity/v3/projects/064ed24d1ffb41e68e6d67f0af0b7413 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:28.603483 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f491803-99a4-46f3-8970-efec4050f314 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:28.604076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f491803-99a4-46f3-8970-efec4050f314 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:28.604196 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f491803-99a4-46f3-8970-efec4050f314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:28.626433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3f491803-99a4-46f3-8970-efec4050f314 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:28.627735 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3f491803-99a4-46f3-8970-efec4050f314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:28.689273 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1314/2612] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:28.697745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd603fdc-2715-47a6-bb8d-29be3849cb33 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:28.698049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd603fdc-2715-47a6-bb8d-29be3849cb33 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:28.698328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd603fdc-2715-47a6-bb8d-29be3849cb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:28.717207 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-bd603fdc-2715-47a6-bb8d-29be3849cb33 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:28.718342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-bd603fdc-2715-47a6-bb8d-29be3849cb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:28.726647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1299/2613] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:28.732538 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f491803-99a4-46f3-8970-efec4050f314 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:28.742977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f491803-99a4-46f3-8970-efec4050f314 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:28.749794 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a3173-2ff0-4421-8733-b2fdf3e01077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:28.750474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a3173-2ff0-4421-8733-b2fdf3e01077 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:28.750788 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a3173-2ff0-4421-8733-b2fdf3e01077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:28.751063 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a3173-2ff0-4421-8733-b2fdf3e01077 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:28.751787 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a3173-2ff0-4421-8733-b2fdf3e01077 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:38:28.752093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a3173-2ff0-4421-8733-b2fdf3e01077 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:28.753719 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a3173-2ff0-4421-8733-b2fdf3e01077 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:28.767618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1315/2614] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:38:28.772021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd603fdc-2715-47a6-bb8d-29be3849cb33 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:28.782140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd603fdc-2715-47a6-bb8d-29be3849cb33 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:28.786780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f2407-6f7e-4911-a8b4-dc885c20f286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:28.787438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f2407-6f7e-4911-a8b4-dc885c20f286 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:28.787728 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f2407-6f7e-4911-a8b4-dc885c20f286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:28.788009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f2407-6f7e-4911-a8b4-dc885c20f286 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:28.788713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f2407-6f7e-4911-a8b4-dc885c20f286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:28.790923 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f2407-6f7e-4911-a8b4-dc885c20f286 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:28.817950 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1300/2615] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:28 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:28.822948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a3173-2ff0-4421-8733-b2fdf3e01077 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:28.833874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a3173-2ff0-4421-8733-b2fdf3e01077 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:28.894646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282e05a-eedc-49b5-b3fe-3da970bd6024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:28.895317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282e05a-eedc-49b5-b3fe-3da970bd6024 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:28.895599 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282e05a-eedc-49b5-b3fe-3da970bd6024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:28.895876 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282e05a-eedc-49b5-b3fe-3da970bd6024 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:28.896568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282e05a-eedc-49b5-b3fe-3da970bd6024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:28.898504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282e05a-eedc-49b5-b3fe-3da970bd6024 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:28.919115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1316/2616] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:28 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:28.923293 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f2407-6f7e-4911-a8b4-dc885c20f286 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:28.933962 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f2407-6f7e-4911-a8b4-dc885c20f286 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:28.938480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c205a-70d4-46e8-b4b1-aa8fe8eac535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:28.939166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c205a-70d4-46e8-b4b1-aa8fe8eac535 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:28.939456 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c205a-70d4-46e8-b4b1-aa8fe8eac535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:28.939737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c205a-70d4-46e8-b4b1-aa8fe8eac535 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:28.940461 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c205a-70d4-46e8-b4b1-aa8fe8eac535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:28.942067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c205a-70d4-46e8-b4b1-aa8fe8eac535 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:28.952474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1301/2617] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:28.956301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282e05a-eedc-49b5-b3fe-3da970bd6024 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:28.966903 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282e05a-eedc-49b5-b3fe-3da970bd6024 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:28.971643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d34df8-207f-449f-9437-1d766599802e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:28.972268 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d34df8-207f-449f-9437-1d766599802e admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:28.972553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d34df8-207f-449f-9437-1d766599802e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:28.972830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d34df8-207f-449f-9437-1d766599802e admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:28.973555 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d34df8-207f-449f-9437-1d766599802e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:28.975187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d34df8-207f-449f-9437-1d766599802e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:28.985829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1317/2618] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:28.989970 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c205a-70d4-46e8-b4b1-aa8fe8eac535 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:29.000238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c205a-70d4-46e8-b4b1-aa8fe8eac535 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:29.004985 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfe2b3-101c-4399-b23e-6cbf0043f2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:29.005791 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfe2b3-101c-4399-b23e-6cbf0043f2a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:29.006079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfe2b3-101c-4399-b23e-6cbf0043f2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:29.006358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfe2b3-101c-4399-b23e-6cbf0043f2a1 admin admin] PATH_INFO: `/v3/projects/02de469128b44ac4924b1f5aa7cb6ddb/users/8728f2d0b848424d86b24c794c8d3388/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:29.022250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdfe2b3-101c-4399-b23e-6cbf0043f2a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02de469128b44ac4924b1f5aa7cb6ddb, user_id=8728f2d0b848424d86b24c794c8d3388, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:29.026152 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdfe2b3-101c-4399-b23e-6cbf0043f2a1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:29.039197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1302/2619] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:28 2022] PUT /identity/v3/projects/02de469128b44ac4924b1f5aa7cb6ddb/users/8728f2d0b848424d86b24c794c8d3388/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:29.104446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635ad15-b9f3-4070-a0b1-9185a2315601 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:29.104843 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635ad15-b9f3-4070-a0b1-9185a2315601 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:29.105187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635ad15-b9f3-4070-a0b1-9185a2315601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:29.132653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9635ad15-b9f3-4070-a0b1-9185a2315601 None None] MFA Rules not processed for user `8728f2d0b848424d86b24c794c8d3388`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:29.134468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9635ad15-b9f3-4070-a0b1-9185a2315601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:29.142981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:29.202672 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1318/2620] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:29 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:29.214869 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfe2b3-101c-4399-b23e-6cbf0043f2a1 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:29.216586 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6cdfe2b3-101c-4399-b23e-6cbf0043f2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:29.278670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfe2b3-101c-4399-b23e-6cbf0043f2a1 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:29.282322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf836c5-ed35-461d-9e40-9937ee8f69cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:29.283017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf836c5-ed35-461d-9e40-9937ee8f69cc service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:29.283309 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf836c5-ed35-461d-9e40-9937ee8f69cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:29.283591 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf836c5-ed35-461d-9e40-9937ee8f69cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:29.293170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf836c5-ed35-461d-9e40-9937ee8f69cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:29.294659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf836c5-ed35-461d-9e40-9937ee8f69cc service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:29.304302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1303/2621] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:38:39.199492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=23 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:39.300010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=56,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:48.949213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635ad15-b9f3-4070-a0b1-9185a2315601 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:48.950737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9635ad15-b9f3-4070-a0b1-9185a2315601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:49.014248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635ad15-b9f3-4070-a0b1-9185a2315601 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:49.018482 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0f096-d111-406b-89f1-5ac4fc4fc4b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:49.019171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0f096-d111-406b-89f1-5ac4fc4fc4b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:49.019573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0f096-d111-406b-89f1-5ac4fc4fc4b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:49.019852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0f096-d111-406b-89f1-5ac4fc4fc4b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:49.021995 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dcc0f096-d111-406b-89f1-5ac4fc4fc4b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:49.096670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0f096-d111-406b-89f1-5ac4fc4fc4b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:49.098048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0f096-d111-406b-89f1-5ac4fc4fc4b8 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:49.108171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1319/2622] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:52.773678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf836c5-ed35-461d-9e40-9937ee8f69cc service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:52.776129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8bf836c5-ed35-461d-9e40-9937ee8f69cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:52.797188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf836c5-ed35-461d-9e40-9937ee8f69cc service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:52.801656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc12e6-f752-4ac6-8b89-ef88e4731168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:52.802552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc12e6-f752-4ac6-8b89-ef88e4731168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:52.802929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc12e6-f752-4ac6-8b89-ef88e4731168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:52.803211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc12e6-f752-4ac6-8b89-ef88e4731168 admin admin] PATH_INFO: `/v3/users/fb5f0f7e59dd4c548a6ea72bb25f1ebb` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:52.804438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc12e6-f752-4ac6-8b89-ef88e4731168 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb5f0f7e59dd4c548a6ea72bb25f1ebb)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:52.806956 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc12e6-f752-4ac6-8b89-ef88e4731168 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:52.867800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a1dc12e6-f752-4ac6-8b89-ef88e4731168 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb5f0f7e59dd4c548a6ea72bb25f1ebb'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:52.878104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a1dc12e6-f752-4ac6-8b89-ef88e4731168 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb5f0f7e59dd4c548a6ea72bb25f1ebb'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:52.886569 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a1dc12e6-f752-4ac6-8b89-ef88e4731168 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb5f0f7e59dd4c548a6ea72bb25f1ebb'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:52.905030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a1dc12e6-f752-4ac6-8b89-ef88e4731168 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb5f0f7e59dd4c548a6ea72bb25f1ebb'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:52.906160 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a1dc12e6-f752-4ac6-8b89-ef88e4731168 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb5f0f7e59dd4c548a6ea72bb25f1ebb'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:52.917476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1304/2623] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:52 2022] DELETE /identity/v3/users/fb5f0f7e59dd4c548a6ea72bb25f1ebb => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:53.112065 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0f096-d111-406b-89f1-5ac4fc4fc4b8 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:53.127310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0f096-d111-406b-89f1-5ac4fc4fc4b8 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:53.201657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:53.202540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:53.202856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:53.203136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] PATH_INFO: `/v3/projects/80faf6362e244bb992ffb6058ceeefda` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:53.204340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80faf6362e244bb992ffb6058ceeefda)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:53.206308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:53.268089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80faf6362e244bb992ffb6058ceeefda', 'request_id': 'req-188c9d17-ec39-43e5-9864-6f279e3dd1d5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:53.271535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:53.282708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80faf6362e244bb992ffb6058ceeefda', 'request_id': 'req-188c9d17-ec39-43e5-9864-6f279e3dd1d5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:53.290718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80faf6362e244bb992ffb6058ceeefda', 'request_id': 'req-188c9d17-ec39-43e5-9864-6f279e3dd1d5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:53.301402 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] The token cache is being invalidate because project 80faf6362e244bb992ffb6058ceeefda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:53.301694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:53.302356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-188c9d17-ec39-43e5-9864-6f279e3dd1d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:53.303973 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1320/2624] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:53 2022] DELETE /identity/v3/projects/80faf6362e244bb992ffb6058ceeefda => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:53.309427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857776b-e086-478d-9bc1-13c9fb4d02c9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:53.309825 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857776b-e086-478d-9bc1-13c9fb4d02c9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:53.310200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857776b-e086-478d-9bc1-13c9fb4d02c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:53.329463 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0857776b-e086-478d-9bc1-13c9fb4d02c9 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:53.330753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0857776b-e086-478d-9bc1-13c9fb4d02c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:53.405855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1305/2625] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:53.413912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2076cbc-3d14-4c69-a736-681df6db95f8 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:53.414205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2076cbc-3d14-4c69-a736-681df6db95f8 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:53.414495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2076cbc-3d14-4c69-a736-681df6db95f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:53.435548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e2076cbc-3d14-4c69-a736-681df6db95f8 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:53.436814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e2076cbc-3d14-4c69-a736-681df6db95f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:53.445266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1321/2626] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:53.450273 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857776b-e086-478d-9bc1-13c9fb4d02c9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:53.463407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857776b-e086-478d-9bc1-13c9fb4d02c9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:53.469355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c50f73-e3f5-4cdd-97a4-192653a5da2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:53.470223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c50f73-e3f5-4cdd-97a4-192653a5da2a admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:53.470509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c50f73-e3f5-4cdd-97a4-192653a5da2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:53.470818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c50f73-e3f5-4cdd-97a4-192653a5da2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:53.471565 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c50f73-e3f5-4cdd-97a4-192653a5da2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:38:53.471875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c50f73-e3f5-4cdd-97a4-192653a5da2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:53.473498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c50f73-e3f5-4cdd-97a4-192653a5da2a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:53.487887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1306/2627] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:38:53.494054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2076cbc-3d14-4c69-a736-681df6db95f8 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:53.510151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2076cbc-3d14-4c69-a736-681df6db95f8 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:53.518748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bb948-7ad7-459f-96a8-8a6490853007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:53.519868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bb948-7ad7-459f-96a8-8a6490853007 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:53.520412 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bb948-7ad7-459f-96a8-8a6490853007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:53.520741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bb948-7ad7-459f-96a8-8a6490853007 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:53.522175 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5bb948-7ad7-459f-96a8-8a6490853007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:53.524477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5bb948-7ad7-459f-96a8-8a6490853007 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:53.560324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1322/2628] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:53 2022] POST /identity/v3/projects => generated 403 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:53.565344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c50f73-e3f5-4cdd-97a4-192653a5da2a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:53.577345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c50f73-e3f5-4cdd-97a4-192653a5da2a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:53.656620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e39d72-2505-4c88-83a6-dac50fd4ab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:53.657322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e39d72-2505-4c88-83a6-dac50fd4ab59 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:53.657606 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e39d72-2505-4c88-83a6-dac50fd4ab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:53.658006 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e39d72-2505-4c88-83a6-dac50fd4ab59 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:53.658759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e39d72-2505-4c88-83a6-dac50fd4ab59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:53.662091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e39d72-2505-4c88-83a6-dac50fd4ab59 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:53.670088 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:53.688859 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1307/2629] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:53 2022] POST /identity/v3/users => generated 302 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:53.693859 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bb948-7ad7-459f-96a8-8a6490853007 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:53.706159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bb948-7ad7-459f-96a8-8a6490853007 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:53.711675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2352de-7480-430b-99b0-9f0037929cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:53.712857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2352de-7480-430b-99b0-9f0037929cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:53.713144 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2352de-7480-430b-99b0-9f0037929cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:53.713421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2352de-7480-430b-99b0-9f0037929cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:53.714202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2352de-7480-430b-99b0-9f0037929cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:53.716093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2352de-7480-430b-99b0-9f0037929cd3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:53.734708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1323/2630] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:53.736172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e39d72-2505-4c88-83a6-dac50fd4ab59 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:53.747648 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e39d72-2505-4c88-83a6-dac50fd4ab59 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:53.752500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59524e49-0fba-46f3-a806-39e48acba877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:53.753179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59524e49-0fba-46f3-a806-39e48acba877 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:53.753462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59524e49-0fba-46f3-a806-39e48acba877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:53.753739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59524e49-0fba-46f3-a806-39e48acba877 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:53.754525 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59524e49-0fba-46f3-a806-39e48acba877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:53.756239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59524e49-0fba-46f3-a806-39e48acba877 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:53.771385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1308/2631] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:53.775838 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2352de-7480-430b-99b0-9f0037929cd3 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:53.787547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2352de-7480-430b-99b0-9f0037929cd3 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:53.800579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af977df3-fc18-49d3-aa2e-8fbf2dd8d438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:53.801503 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977df3-fc18-49d3-aa2e-8fbf2dd8d438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:53.801788 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977df3-fc18-49d3-aa2e-8fbf2dd8d438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:53.802068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977df3-fc18-49d3-aa2e-8fbf2dd8d438 admin admin] PATH_INFO: `/v3/projects/457c11a970634e409dc20e5d78e7f820/users/18a4c8aa256c4b4ab9e6961720198629/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:53.819200 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af977df3-fc18-49d3-aa2e-8fbf2dd8d438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=457c11a970634e409dc20e5d78e7f820, user_id=18a4c8aa256c4b4ab9e6961720198629, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:53.823406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af977df3-fc18-49d3-aa2e-8fbf2dd8d438 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:53.842456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1324/2632] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:53 2022] PUT /identity/v3/projects/457c11a970634e409dc20e5d78e7f820/users/18a4c8aa256c4b4ab9e6961720198629/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:53.854883 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356688d-fd4c-4ac9-a748-e88754e20a1a None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:53.855711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356688d-fd4c-4ac9-a748-e88754e20a1a None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:53.855711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356688d-fd4c-4ac9-a748-e88754e20a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:53.892222 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-8356688d-fd4c-4ac9-a748-e88754e20a1a None None] MFA Rules not processed for user `18a4c8aa256c4b4ab9e6961720198629`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:53.893536 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-8356688d-fd4c-4ac9-a748-e88754e20a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:53.975087 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1309/2633] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:53.979819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af977df3-fc18-49d3-aa2e-8fbf2dd8d438 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:54.067364 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af977df3-fc18-49d3-aa2e-8fbf2dd8d438 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:54.073409 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6574e1-5f59-4611-8578-76f27afc9863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:54.074202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6574e1-5f59-4611-8578-76f27afc9863 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:54.074500 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6574e1-5f59-4611-8578-76f27afc9863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:54.074877 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6574e1-5f59-4611-8578-76f27afc9863 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:54.075818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6574e1-5f59-4611-8578-76f27afc9863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:54.078369 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6574e1-5f59-4611-8578-76f27afc9863 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:54.115484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1325/2634] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:53 2022] POST /identity/v3/projects => generated 403 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:54.116796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356688d-fd4c-4ac9-a748-e88754e20a1a None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:54.129392 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356688d-fd4c-4ac9-a748-e88754e20a1a None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:54.195405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dcd99-5826-4233-bd0b-0cb17a452eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:54.196509 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dcd99-5826-4233-bd0b-0cb17a452eec admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:54.196792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dcd99-5826-4233-bd0b-0cb17a452eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:54.197066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938dcd99-5826-4233-bd0b-0cb17a452eec admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:54.197774 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dcd99-5826-4233-bd0b-0cb17a452eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:54.199738 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938dcd99-5826-4233-bd0b-0cb17a452eec admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:54.223036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1310/2635] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:54 2022] POST /identity/v3/users => generated 308 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:38:54.227866 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6574e1-5f59-4611-8578-76f27afc9863 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:54.238889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6574e1-5f59-4611-8578-76f27afc9863 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:54.244425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a9ace-3c84-490e-b951-9f7b9ccc9f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:54.245105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a9ace-3c84-490e-b951-9f7b9ccc9f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:54.245467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a9ace-3c84-490e-b951-9f7b9ccc9f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:54.246296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a9ace-3c84-490e-b951-9f7b9ccc9f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:54.247065 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a9ace-3c84-490e-b951-9f7b9ccc9f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:54.248674 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a9ace-3c84-490e-b951-9f7b9ccc9f21 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:54.261362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1326/2636] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:38:54.265913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dcd99-5826-4233-bd0b-0cb17a452eec admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:54.277157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938dcd99-5826-4233-bd0b-0cb17a452eec admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:54.284234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c009d-4fa2-4555-ae63-80c6a69d134c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:54.285306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c009d-4fa2-4555-ae63-80c6a69d134c admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:54.285611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c009d-4fa2-4555-ae63-80c6a69d134c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:54.286017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c009d-4fa2-4555-ae63-80c6a69d134c admin admin] PATH_INFO: `/v3/projects/6812b3569a5b47a1b1d1c8c01120b968/users/6351608bce1d43cb9484a3b38caa39a8/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:54.303998 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c009d-4fa2-4555-ae63-80c6a69d134c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6812b3569a5b47a1b1d1c8c01120b968, user_id=6351608bce1d43cb9484a3b38caa39a8, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:54.308105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c009d-4fa2-4555-ae63-80c6a69d134c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:54.324761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1311/2637] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:54 2022] PUT /identity/v3/projects/6812b3569a5b47a1b1d1c8c01120b968/users/6351608bce1d43cb9484a3b38caa39a8/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:38:54.329173 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a9ace-3c84-490e-b951-9f7b9ccc9f21 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:54.340004 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a9ace-3c84-490e-b951-9f7b9ccc9f21 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:54.395933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:54.413564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a4cb2-7090-4957-9210-6bca822aabe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:54.414705 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a4cb2-7090-4957-9210-6bca822aabe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:54.415197 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a4cb2-7090-4957-9210-6bca822aabe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:54.415912 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a4cb2-7090-4957-9210-6bca822aabe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:54.417296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a4cb2-7090-4957-9210-6bca822aabe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:54.424512 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a4cb2-7090-4957-9210-6bca822aabe6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:54.439937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1327/2638] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:54.444308 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c009d-4fa2-4555-ae63-80c6a69d134c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:54.456027 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c009d-4fa2-4555-ae63-80c6a69d134c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:54.461566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51b132-2394-48b9-bef9-7698741e2dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:54.462460 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51b132-2394-48b9-bef9-7698741e2dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:54.462792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51b132-2394-48b9-bef9-7698741e2dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:54.463138 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51b132-2394-48b9-bef9-7698741e2dfe admin admin] PATH_INFO: `/v3/projects/6812b3569a5b47a1b1d1c8c01120b968/users/6351608bce1d43cb9484a3b38caa39a8/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:54.465389 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51b132-2394-48b9-bef9-7698741e2dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6812b3569a5b47a1b1d1c8c01120b968, user_id=6351608bce1d43cb9484a3b38caa39a8, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:54.468535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51b132-2394-48b9-bef9-7698741e2dfe admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:54.481256 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1312/2639] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:54 2022] PUT /identity/v3/projects/6812b3569a5b47a1b1d1c8c01120b968/users/6351608bce1d43cb9484a3b38caa39a8/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:54.493267 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30561607-6b32-4d87-a9e3-48e0d6ad12cb None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:54.493533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30561607-6b32-4d87-a9e3-48e0d6ad12cb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:54.493811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30561607-6b32-4d87-a9e3-48e0d6ad12cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:54.528078 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-30561607-6b32-4d87-a9e3-48e0d6ad12cb None None] MFA Rules not processed for user `6351608bce1d43cb9484a3b38caa39a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:54.529928 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-30561607-6b32-4d87-a9e3-48e0d6ad12cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:54.620589 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1328/2640] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:54 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:54.632581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51b132-2394-48b9-bef9-7698741e2dfe admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:54.638134 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ec51b132-2394-48b9-bef9-7698741e2dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:54.659294 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:54.704025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51b132-2394-48b9-bef9-7698741e2dfe admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:54.708034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75379fb-8976-4bd6-a306-ba64dc4726f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:54.708761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75379fb-8976-4bd6-a306-ba64dc4726f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:54.709050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75379fb-8976-4bd6-a306-ba64dc4726f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:54.709330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75379fb-8976-4bd6-a306-ba64dc4726f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:54.720639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75379fb-8976-4bd6-a306-ba64dc4726f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:54.722072 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75379fb-8976-4bd6-a306-ba64dc4726f4 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:54.732301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1313/2641] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:54 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:54.887974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30561607-6b32-4d87-a9e3-48e0d6ad12cb None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:54.957279 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30561607-6b32-4d87-a9e3-48e0d6ad12cb None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:54.962283 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:54.963316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:54.963567 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:54.963808 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] PATH_INFO: `/v3/users/18a4c8aa256c4b4ab9e6961720198629` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:54.965117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18a4c8aa256c4b4ab9e6961720198629)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:54.967263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:55.021254 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18a4c8aa256c4b4ab9e6961720198629'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.034341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18a4c8aa256c4b4ab9e6961720198629'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.035285 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18a4c8aa256c4b4ab9e6961720198629'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.043502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18a4c8aa256c4b4ab9e6961720198629'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.046001 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:55.053654 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18a4c8aa256c4b4ab9e6961720198629'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.061081 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1329/2642] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:54 2022] DELETE /identity/v3/users/18a4c8aa256c4b4ab9e6961720198629 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:55.065982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fbaff-e7f1-4c60-ae7c-9630c4e36285 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:55.066512 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fbaff-e7f1-4c60-ae7c-9630c4e36285 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:55.066651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fbaff-e7f1-4c60-ae7c-9630c4e36285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:55.084289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b26fbaff-e7f1-4c60-ae7c-9630c4e36285 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:55.085514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b26fbaff-e7f1-4c60-ae7c-9630c4e36285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:55.158239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1314/2643] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:55.170602 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:55.172067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:55.236161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1bbdc1-6e24-49b6-9852-9f0409d05636 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:55.240172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0f11b-a8b8-4554-a369-91e652d3faee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:55.240830 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0f11b-a8b8-4554-a369-91e652d3faee service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:55.241139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0f11b-a8b8-4554-a369-91e652d3faee service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:55.241561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0f11b-a8b8-4554-a369-91e652d3faee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:55.251491 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0f11b-a8b8-4554-a369-91e652d3faee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:55.252944 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0f11b-a8b8-4554-a369-91e652d3faee service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:55.262921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1330/2644] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:55.561994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fbaff-e7f1-4c60-ae7c-9630c4e36285 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:55.561994 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b26fbaff-e7f1-4c60-ae7c-9630c4e36285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:55.577080 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fbaff-e7f1-4c60-ae7c-9630c4e36285 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:55.581947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:55.583415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:55.583787 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:55.584140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] PATH_INFO: `/v3/projects/457c11a970634e409dc20e5d78e7f820` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:55.585532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=457c11a970634e409dc20e5d78e7f820)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:55.587559 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:55.640163 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '457c11a970634e409dc20e5d78e7f820', 'request_id': 'req-43136002-f167-45e2-b0c6-392794d5f2e1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.645815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '457c11a970634e409dc20e5d78e7f820', 'request_id': 'req-43136002-f167-45e2-b0c6-392794d5f2e1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.653057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '457c11a970634e409dc20e5d78e7f820', 'request_id': 'req-43136002-f167-45e2-b0c6-392794d5f2e1'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.667183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] The token cache is being invalidate because project 457c11a970634e409dc20e5d78e7f820 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:55.667481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.668164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.669739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1315/2645] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:55 2022] DELETE /identity/v3/projects/457c11a970634e409dc20e5d78e7f820 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:55.674282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0f11b-a8b8-4554-a369-91e652d3faee service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:55.684771 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0f11b-a8b8-4554-a369-91e652d3faee service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:55.747411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c672a-f2c2-4f89-b593-238d910a9360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:55.748219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c672a-f2c2-4f89-b593-238d910a9360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:55.748507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c672a-f2c2-4f89-b593-238d910a9360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:55.748815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c672a-f2c2-4f89-b593-238d910a9360 admin admin] PATH_INFO: `/v3/users/6351608bce1d43cb9484a3b38caa39a8` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:55.750082 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6c672a-f2c2-4f89-b593-238d910a9360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6351608bce1d43cb9484a3b38caa39a8)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:55.752067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6c672a-f2c2-4f89-b593-238d910a9360 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:55.805100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ad6c672a-f2c2-4f89-b593-238d910a9360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6351608bce1d43cb9484a3b38caa39a8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.818246 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ad6c672a-f2c2-4f89-b593-238d910a9360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6351608bce1d43cb9484a3b38caa39a8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.818971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ad6c672a-f2c2-4f89-b593-238d910a9360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6351608bce1d43cb9484a3b38caa39a8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.829693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ad6c672a-f2c2-4f89-b593-238d910a9360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6351608bce1d43cb9484a3b38caa39a8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.838504 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ad6c672a-f2c2-4f89-b593-238d910a9360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6351608bce1d43cb9484a3b38caa39a8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:55.845365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1331/2646] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:38:55 2022] DELETE /identity/v3/users/6351608bce1d43cb9484a3b38caa39a8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:56.150847 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:56.152897 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:56.167670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43136002-f167-45e2-b0c6-392794d5f2e1 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:56.221201 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:56.241230 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:56.242404 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:56.242763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:56.243063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] PATH_INFO: `/v3/projects/6812b3569a5b47a1b1d1c8c01120b968` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:56.245217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6812b3569a5b47a1b1d1c8c01120b968)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:56.247364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:56.303209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6812b3569a5b47a1b1d1c8c01120b968', 'request_id': 'req-54a7a5c1-3100-4a83-8561-4a3f1572cda7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:56.309607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6812b3569a5b47a1b1d1c8c01120b968', 'request_id': 'req-54a7a5c1-3100-4a83-8561-4a3f1572cda7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:56.317946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6812b3569a5b47a1b1d1c8c01120b968', 'request_id': 'req-54a7a5c1-3100-4a83-8561-4a3f1572cda7'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:56.338951 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] The token cache is being invalidate because project 6812b3569a5b47a1b1d1c8c01120b968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:38:56.339265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:56.339976 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-54a7a5c1-3100-4a83-8561-4a3f1572cda7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:38:56.341705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1316/2647] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:38:56 2022] DELETE /identity/v3/projects/6812b3569a5b47a1b1d1c8c01120b968 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:56.347778 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3047c60-b9b7-4228-8e9c-a17b08044327 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:56.348418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3047c60-b9b7-4228-8e9c-a17b08044327 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:56.348418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3047c60-b9b7-4228-8e9c-a17b08044327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:56.371537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-e3047c60-b9b7-4228-8e9c-a17b08044327 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:56.372890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e3047c60-b9b7-4228-8e9c-a17b08044327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:56.383323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:56.462357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1332/2648] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:56.471730 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48326-07be-4b09-828a-964aa23c714c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:56.472108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48326-07be-4b09-828a-964aa23c714c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:56.472553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd48326-07be-4b09-828a-964aa23c714c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:56.496849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7cd48326-07be-4b09-828a-964aa23c714c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:56.498132 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7cd48326-07be-4b09-828a-964aa23c714c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:56.507602 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1317/2649] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:56.513991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3047c60-b9b7-4228-8e9c-a17b08044327 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:56.526536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3047c60-b9b7-4228-8e9c-a17b08044327 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:56.532691 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb7270-9f75-4f7f-ad66-b8ae6f9694a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:56.533378 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb7270-9f75-4f7f-ad66-b8ae6f9694a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:56.533779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb7270-9f75-4f7f-ad66-b8ae6f9694a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:56.534226 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb7270-9f75-4f7f-ad66-b8ae6f9694a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:56.535053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb7270-9f75-4f7f-ad66-b8ae6f9694a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:38:56.535836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb7270-9f75-4f7f-ad66-b8ae6f9694a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:56.537153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb7270-9f75-4f7f-ad66-b8ae6f9694a2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:56.561685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1333/2650] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:38:56.563538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd48326-07be-4b09-828a-964aa23c714c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:56.574904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd48326-07be-4b09-828a-964aa23c714c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:56.583235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8792-694c-458d-9d6d-fc0a570f5653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:56.583235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8792-694c-458d-9d6d-fc0a570f5653 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:56.583235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8792-694c-458d-9d6d-fc0a570f5653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:56.583235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8792-694c-458d-9d6d-fc0a570f5653 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:56.583792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e8792-694c-458d-9d6d-fc0a570f5653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:56.585016 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e8792-694c-458d-9d6d-fc0a570f5653 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:56.618613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1318/2651] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:56 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:38:56.623169 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb7270-9f75-4f7f-ad66-b8ae6f9694a2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:56.636624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb7270-9f75-4f7f-ad66-b8ae6f9694a2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:56.709679 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429ee5d-7fda-4083-ba0b-44e0e2c116ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:56.710375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429ee5d-7fda-4083-ba0b-44e0e2c116ea admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:56.710692 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429ee5d-7fda-4083-ba0b-44e0e2c116ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:56.710977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429ee5d-7fda-4083-ba0b-44e0e2c116ea admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:56.711697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429ee5d-7fda-4083-ba0b-44e0e2c116ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:56.714391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429ee5d-7fda-4083-ba0b-44e0e2c116ea admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:56.740989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1334/2652] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:56 2022] POST /identity/v3/users => generated 303 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:56.745953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8792-694c-458d-9d6d-fc0a570f5653 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:56.758411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8792-694c-458d-9d6d-fc0a570f5653 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:56.763617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272dc4a9-dc88-430d-9ff2-c884a14b2755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:56.764403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dc4a9-dc88-430d-9ff2-c884a14b2755 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:56.765244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dc4a9-dc88-430d-9ff2-c884a14b2755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:56.765531 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272dc4a9-dc88-430d-9ff2-c884a14b2755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:56.766377 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272dc4a9-dc88-430d-9ff2-c884a14b2755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:56.768007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272dc4a9-dc88-430d-9ff2-c884a14b2755 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:56.781753 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1319/2653] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:38:56.784541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429ee5d-7fda-4083-ba0b-44e0e2c116ea admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:56.795956 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429ee5d-7fda-4083-ba0b-44e0e2c116ea admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:56.810697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d33ad2-42a3-4b07-952e-a3e58f048f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:56.812874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d33ad2-42a3-4b07-952e-a3e58f048f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:56.813100 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d33ad2-42a3-4b07-952e-a3e58f048f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:56.813376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d33ad2-42a3-4b07-952e-a3e58f048f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:56.814219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d33ad2-42a3-4b07-952e-a3e58f048f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:56.818006 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d33ad2-42a3-4b07-952e-a3e58f048f58 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:56.833971 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1335/2654] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:56.838613 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272dc4a9-dc88-430d-9ff2-c884a14b2755 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:56.851815 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272dc4a9-dc88-430d-9ff2-c884a14b2755 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:56.856123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5a42a-0fe9-4478-9880-fd659e697b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:56.857052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5a42a-0fe9-4478-9880-fd659e697b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:56.857340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5a42a-0fe9-4478-9880-fd659e697b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:56.857777 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a5a42a-0fe9-4478-9880-fd659e697b34 admin admin] PATH_INFO: `/v3/projects/6f30a3d0ea32493c8279de7ec5fc6802/users/cda6c3ffb1034aa09b9fb4f81a36fc1c/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:56.875782 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5a42a-0fe9-4478-9880-fd659e697b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f30a3d0ea32493c8279de7ec5fc6802, user_id=cda6c3ffb1034aa09b9fb4f81a36fc1c, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:56.878490 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a5a42a-0fe9-4478-9880-fd659e697b34 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:56.897607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1320/2655] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:56 2022] PUT /identity/v3/projects/6f30a3d0ea32493c8279de7ec5fc6802/users/cda6c3ffb1034aa09b9fb4f81a36fc1c/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:56.906751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de3424-785d-44ef-867c-7407bd300ab3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:56.907773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de3424-785d-44ef-867c-7407bd300ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:56.907773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de3424-785d-44ef-867c-7407bd300ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:56.943473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-05de3424-785d-44ef-867c-7407bd300ab3 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:56.946404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-05de3424-785d-44ef-867c-7407bd300ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:57.032856 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1336/2656] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:57.045642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5a42a-0fe9-4478-9880-fd659e697b34 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:57.048834 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-18a5a42a-0fe9-4478-9880-fd659e697b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:57.118157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a5a42a-0fe9-4478-9880-fd659e697b34 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:57.125401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40525744-48b6-4f57-8cd6-20f5bfdfe143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:57.126108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40525744-48b6-4f57-8cd6-20f5bfdfe143 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:57.126393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40525744-48b6-4f57-8cd6-20f5bfdfe143 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:57.126703 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40525744-48b6-4f57-8cd6-20f5bfdfe143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:57.146043 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40525744-48b6-4f57-8cd6-20f5bfdfe143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:57.147587 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40525744-48b6-4f57-8cd6-20f5bfdfe143 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:57.159526 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1321/2657] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:38:58.313759 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36213380-8a80-44ff-ad75-efdcc64059b2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:58.314411 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36213380-8a80-44ff-ad75-efdcc64059b2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:58.315085 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36213380-8a80-44ff-ad75-efdcc64059b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:58.363506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-36213380-8a80-44ff-ad75-efdcc64059b2 None None] MFA Rules not processed for user `cda6c3ffb1034aa09b9fb4f81a36fc1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:58.364800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-36213380-8a80-44ff-ad75-efdcc64059b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:58.373935 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:58.442852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1337/2658] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:58 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:38:58.446481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40525744-48b6-4f57-8cd6-20f5bfdfe143 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:58.458033 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40525744-48b6-4f57-8cd6-20f5bfdfe143 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:58.466880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e301d-5ae3-4c80-ac29-7b1273f923af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:58.467936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e301d-5ae3-4c80-ac29-7b1273f923af admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:58.468260 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e301d-5ae3-4c80-ac29-7b1273f923af admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:58.469037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e301d-5ae3-4c80-ac29-7b1273f923af admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:58.469973 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e301d-5ae3-4c80-ac29-7b1273f923af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:58.472202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e301d-5ae3-4c80-ac29-7b1273f923af admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:58.489452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:38:58.508875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1322/2659] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:38:58 2022] POST /identity/v3/projects => generated 405 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:58.514034 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36213380-8a80-44ff-ad75-efdcc64059b2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:58.530398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36213380-8a80-44ff-ad75-efdcc64059b2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:58.606395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95125a89-0236-417b-b693-d320222d9e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:58.607114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95125a89-0236-417b-b693-d320222d9e47 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:58.607399 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95125a89-0236-417b-b693-d320222d9e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:58.607677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95125a89-0236-417b-b693-d320222d9e47 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:58.608377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95125a89-0236-417b-b693-d320222d9e47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:58.610472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95125a89-0236-417b-b693-d320222d9e47 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:58.647148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1338/2660] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:38:58 2022] POST /identity/v3/users => generated 309 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:38:58.651021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e301d-5ae3-4c80-ac29-7b1273f923af admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:58.662274 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e301d-5ae3-4c80-ac29-7b1273f923af admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:58.668199 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ff468-0d20-4c72-98a9-3b5419993477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:58.669750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ff468-0d20-4c72-98a9-3b5419993477 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:58.670133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ff468-0d20-4c72-98a9-3b5419993477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:58.670528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ff468-0d20-4c72-98a9-3b5419993477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:58.671500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ff468-0d20-4c72-98a9-3b5419993477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:58.673191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ff468-0d20-4c72-98a9-3b5419993477 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:58.687170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1323/2661] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:58.692310 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95125a89-0236-417b-b693-d320222d9e47 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:58.710383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95125a89-0236-417b-b693-d320222d9e47 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:58.717327 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f45bb-b432-4f4f-8ac6-6ea82674337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:58.719064 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f45bb-b432-4f4f-8ac6-6ea82674337a admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:58.719425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f45bb-b432-4f4f-8ac6-6ea82674337a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:58.719729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f45bb-b432-4f4f-8ac6-6ea82674337a admin admin] PATH_INFO: `/v3/projects/93d48ce74eb949f7973a2b91e1b84772/users/850638b6c4114628b08bad705ac7b323/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:58.741968 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f45bb-b432-4f4f-8ac6-6ea82674337a admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d48ce74eb949f7973a2b91e1b84772, user_id=850638b6c4114628b08bad705ac7b323, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:58.745542 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f45bb-b432-4f4f-8ac6-6ea82674337a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:58.760060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1339/2662] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:58 2022] PUT /identity/v3/projects/93d48ce74eb949f7973a2b91e1b84772/users/850638b6c4114628b08bad705ac7b323/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:58.764385 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ff468-0d20-4c72-98a9-3b5419993477 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:58.776543 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ff468-0d20-4c72-98a9-3b5419993477 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:58.839813 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc2ee50-cb3f-4d9c-8dbf-b64718ab39a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:58.840858 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2ee50-cb3f-4d9c-8dbf-b64718ab39a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:58.841184 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2ee50-cb3f-4d9c-8dbf-b64718ab39a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:58.841467 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc2ee50-cb3f-4d9c-8dbf-b64718ab39a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:58.842246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2ee50-cb3f-4d9c-8dbf-b64718ab39a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:58.843935 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc2ee50-cb3f-4d9c-8dbf-b64718ab39a4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:58.855899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1324/2663] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:38:58 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:38:58.860374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f45bb-b432-4f4f-8ac6-6ea82674337a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:38:58.872496 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f45bb-b432-4f4f-8ac6-6ea82674337a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:38:58.877423 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42f7cc-85dc-4b00-856d-36d8c2ec1c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:38:58.878328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f7cc-85dc-4b00-856d-36d8c2ec1c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:58.878637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f7cc-85dc-4b00-856d-36d8c2ec1c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:58.878919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f7cc-85dc-4b00-856d-36d8c2ec1c95 admin admin] PATH_INFO: `/v3/projects/93d48ce74eb949f7973a2b91e1b84772/users/850638b6c4114628b08bad705ac7b323/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:58.881046 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42f7cc-85dc-4b00-856d-36d8c2ec1c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d48ce74eb949f7973a2b91e1b84772, user_id=850638b6c4114628b08bad705ac7b323, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:38:58.883953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42f7cc-85dc-4b00-856d-36d8c2ec1c95 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:38:58.896130 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1340/2664] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:38:58 2022] PUT /identity/v3/projects/93d48ce74eb949f7973a2b91e1b84772/users/850638b6c4114628b08bad705ac7b323/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:38:59.620646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cbd83-61e1-45f3-8e7d-7984c00f0b5d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:38:59.621213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cbd83-61e1-45f3-8e7d-7984c00f0b5d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:38:59.621213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cbd83-61e1-45f3-8e7d-7984c00f0b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:38:59.651083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-037cbd83-61e1-45f3-8e7d-7984c00f0b5d None None] MFA Rules not processed for user `850638b6c4114628b08bad705ac7b323`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:38:59.652794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-037cbd83-61e1-45f3-8e7d-7984c00f0b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:38:59.759557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1325/2665] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:38:59 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:00.938452 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42f7cc-85dc-4b00-856d-36d8c2ec1c95 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:01.005231 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42f7cc-85dc-4b00-856d-36d8c2ec1c95 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:01.010105 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:01.010820 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:01.011108 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:01.011386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] PATH_INFO: `/v3/users/cda6c3ffb1034aa09b9fb4f81a36fc1c` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:01.012594 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cda6c3ffb1034aa09b9fb4f81a36fc1c)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:01.014807 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:01.017354 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:01.079380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cda6c3ffb1034aa09b9fb4f81a36fc1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.101986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cda6c3ffb1034aa09b9fb4f81a36fc1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.103165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cda6c3ffb1034aa09b9fb4f81a36fc1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.114296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cda6c3ffb1034aa09b9fb4f81a36fc1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.126103 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cda6c3ffb1034aa09b9fb4f81a36fc1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.137755 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1341/2666] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:00 2022] DELETE /identity/v3/users/cda6c3ffb1034aa09b9fb4f81a36fc1c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:01.414131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037cbd83-61e1-45f3-8e7d-7984c00f0b5d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:01.432249 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037cbd83-61e1-45f3-8e7d-7984c00f0b5d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:01.502340 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:01.503142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:01.503430 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:01.503721 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] PATH_INFO: `/v3/projects/6f30a3d0ea32493c8279de7ec5fc6802` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:01.505067 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f30a3d0ea32493c8279de7ec5fc6802)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:01.507174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:01.515139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:01.572649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f30a3d0ea32493c8279de7ec5fc6802', 'request_id': 'req-cab20b31-af7c-4533-85ef-2f71f283acbb'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.579264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f30a3d0ea32493c8279de7ec5fc6802', 'request_id': 'req-cab20b31-af7c-4533-85ef-2f71f283acbb'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.585891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f30a3d0ea32493c8279de7ec5fc6802', 'request_id': 'req-cab20b31-af7c-4533-85ef-2f71f283acbb'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.600298 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] The token cache is being invalidate because project 6f30a3d0ea32493c8279de7ec5fc6802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:01.600591 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.601247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.602872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1326/2667] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:01 2022] DELETE /identity/v3/projects/6f30a3d0ea32493c8279de7ec5fc6802 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:01.607287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:01.608803 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:01.623061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537ceed-432f-4e45-81eb-24d013874eb2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:01.692295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9af49-d64e-4ed2-b765-ccff379f5048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:01.693303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9af49-d64e-4ed2-b765-ccff379f5048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:01.693630 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9af49-d64e-4ed2-b765-ccff379f5048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:01.694290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9af49-d64e-4ed2-b765-ccff379f5048 admin admin] PATH_INFO: `/v3/users/850638b6c4114628b08bad705ac7b323` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:01.695806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9af49-d64e-4ed2-b765-ccff379f5048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=850638b6c4114628b08bad705ac7b323)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:01.699174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9af49-d64e-4ed2-b765-ccff379f5048 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:01.771135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a7e9af49-d64e-4ed2-b765-ccff379f5048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '850638b6c4114628b08bad705ac7b323'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.786250 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a7e9af49-d64e-4ed2-b765-ccff379f5048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '850638b6c4114628b08bad705ac7b323'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.787667 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a7e9af49-d64e-4ed2-b765-ccff379f5048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '850638b6c4114628b08bad705ac7b323'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.798439 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a7e9af49-d64e-4ed2-b765-ccff379f5048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '850638b6c4114628b08bad705ac7b323'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.811406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-a7e9af49-d64e-4ed2-b765-ccff379f5048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '850638b6c4114628b08bad705ac7b323'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:01.816177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:01.826076 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1342/2668] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:01 2022] DELETE /identity/v3/users/850638b6c4114628b08bad705ac7b323 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:02.060192 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:02.061704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:02.081593 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab20b31-af7c-4533-85ef-2f71f283acbb admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:02.150361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:02.151347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:02.151614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:02.151891 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] PATH_INFO: `/v3/projects/93d48ce74eb949f7973a2b91e1b84772` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:02.153143 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93d48ce74eb949f7973a2b91e1b84772)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:02.155209 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:02.208676 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93d48ce74eb949f7973a2b91e1b84772', 'request_id': 'req-45f3ddf7-8400-45c9-a821-7136385cc8c5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:02.215936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93d48ce74eb949f7973a2b91e1b84772', 'request_id': 'req-45f3ddf7-8400-45c9-a821-7136385cc8c5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:02.225089 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93d48ce74eb949f7973a2b91e1b84772', 'request_id': 'req-45f3ddf7-8400-45c9-a821-7136385cc8c5'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:02.239842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] The token cache is being invalidate because project 93d48ce74eb949f7973a2b91e1b84772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:02.240157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:02.240766 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-45f3ddf7-8400-45c9-a821-7136385cc8c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:02.242336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1327/2669] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:02 2022] DELETE /identity/v3/projects/93d48ce74eb949f7973a2b91e1b84772 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:02.247733 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64fc44-8180-4216-a4b6-35e3d7c189eb None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:02.248380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64fc44-8180-4216-a4b6-35e3d7c189eb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:02.248380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64fc44-8180-4216-a4b6-35e3d7c189eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:02.270271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ad64fc44-8180-4216-a4b6-35e3d7c189eb None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:02.271863 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ad64fc44-8180-4216-a4b6-35e3d7c189eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:02.360603 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1343/2670] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:02.370435 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e5421-3739-4ba2-abc9-cf9567ed4d14 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:02.370961 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e5421-3739-4ba2-abc9-cf9567ed4d14 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:02.371318 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2e5421-3739-4ba2-abc9-cf9567ed4d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:02.396499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-dc2e5421-3739-4ba2-abc9-cf9567ed4d14 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:02.396499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dc2e5421-3739-4ba2-abc9-cf9567ed4d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:02.408031 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1328/2671] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:02.413048 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64fc44-8180-4216-a4b6-35e3d7c189eb None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:02.425937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64fc44-8180-4216-a4b6-35e3d7c189eb None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:02.434802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ce605-fb94-4aaa-b9bd-c48dcfb7bf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:02.434802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ce605-fb94-4aaa-b9bd-c48dcfb7bf1b admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:02.434802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ce605-fb94-4aaa-b9bd-c48dcfb7bf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:02.434802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ce605-fb94-4aaa-b9bd-c48dcfb7bf1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:02.435350 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ce605-fb94-4aaa-b9bd-c48dcfb7bf1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:39:02.435521 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ce605-fb94-4aaa-b9bd-c48dcfb7bf1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:02.437712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ce605-fb94-4aaa-b9bd-c48dcfb7bf1b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:02.453704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1344/2672] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:39:02.458682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e5421-3739-4ba2-abc9-cf9567ed4d14 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:02.473538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2e5421-3739-4ba2-abc9-cf9567ed4d14 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:02.481572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31689f82-b522-4ae3-b24f-7a45d16391c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:02.482776 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31689f82-b522-4ae3-b24f-7a45d16391c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:02.483137 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31689f82-b522-4ae3-b24f-7a45d16391c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:02.483521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31689f82-b522-4ae3-b24f-7a45d16391c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:02.484244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31689f82-b522-4ae3-b24f-7a45d16391c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:02.486186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31689f82-b522-4ae3-b24f-7a45d16391c7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:02.502704 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:02.517844 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1329/2673] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:02 2022] POST /identity/v3/projects => generated 409 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:02.522174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ce605-fb94-4aaa-b9bd-c48dcfb7bf1b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:02.533054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ce605-fb94-4aaa-b9bd-c48dcfb7bf1b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:02.609320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8988d62-ac27-4023-80e1-f72b891f428f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:02.609320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8988d62-ac27-4023-80e1-f72b891f428f admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:02.609320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8988d62-ac27-4023-80e1-f72b891f428f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:02.609320 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8988d62-ac27-4023-80e1-f72b891f428f admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:02.610554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8988d62-ac27-4023-80e1-f72b891f428f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:02.610554 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8988d62-ac27-4023-80e1-f72b891f428f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:02.631527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1345/2674] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:02 2022] POST /identity/v3/users => generated 305 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:02.637098 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31689f82-b522-4ae3-b24f-7a45d16391c7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:02.649237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31689f82-b522-4ae3-b24f-7a45d16391c7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:02.655765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a27c7-e161-4bfb-9d40-51f65194b1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:02.656812 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a27c7-e161-4bfb-9d40-51f65194b1cd admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:02.657990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a27c7-e161-4bfb-9d40-51f65194b1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:02.657990 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a27c7-e161-4bfb-9d40-51f65194b1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:02.659071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a27c7-e161-4bfb-9d40-51f65194b1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:02.660287 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a27c7-e161-4bfb-9d40-51f65194b1cd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:02.672212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1330/2675] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:39:02.677119 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8988d62-ac27-4023-80e1-f72b891f428f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:02.688823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8988d62-ac27-4023-80e1-f72b891f428f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:02.694106 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f16af10-dc8a-40db-bb6a-c572419df5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:02.694836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16af10-dc8a-40db-bb6a-c572419df5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:02.695149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16af10-dc8a-40db-bb6a-c572419df5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:02.695427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16af10-dc8a-40db-bb6a-c572419df5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:02.696164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16af10-dc8a-40db-bb6a-c572419df5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:02.698495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f16af10-dc8a-40db-bb6a-c572419df5e7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:02.710053 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1346/2676] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:02.715476 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a27c7-e161-4bfb-9d40-51f65194b1cd admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:02.729784 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a27c7-e161-4bfb-9d40-51f65194b1cd admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:02.734901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da75f2-69fd-439a-9c7a-9112799e27a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:02.735833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da75f2-69fd-439a-9c7a-9112799e27a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:02.736124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da75f2-69fd-439a-9c7a-9112799e27a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:02.736402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da75f2-69fd-439a-9c7a-9112799e27a5 admin admin] PATH_INFO: `/v3/projects/4088e3eced9742acbe941f2324d44d60/users/9565eb195c6044c18a70a079afb779e7/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:02.753636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da75f2-69fd-439a-9c7a-9112799e27a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4088e3eced9742acbe941f2324d44d60, user_id=9565eb195c6044c18a70a079afb779e7, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:02.756419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da75f2-69fd-439a-9c7a-9112799e27a5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:02.774345 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1331/2677] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:02 2022] PUT /identity/v3/projects/4088e3eced9742acbe941f2324d44d60/users/9565eb195c6044c18a70a079afb779e7/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:02.785147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0ad4b-0d11-495c-b451-08010457a4ab None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:02.785443 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0ad4b-0d11-495c-b451-08010457a4ab None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:02.785729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0ad4b-0d11-495c-b451-08010457a4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:02.815626 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ebc0ad4b-0d11-495c-b451-08010457a4ab None None] MFA Rules not processed for user `9565eb195c6044c18a70a079afb779e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:02.816899 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ebc0ad4b-0d11-495c-b451-08010457a4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:02.887865 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:02.901011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1347/2678] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:02.906633 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da75f2-69fd-439a-9c7a-9112799e27a5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:02.979763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da75f2-69fd-439a-9c7a-9112799e27a5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:02.984545 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811d221-ec85-4e71-b9a4-728011bfe8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:02.985242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811d221-ec85-4e71-b9a4-728011bfe8aa admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:02.985521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811d221-ec85-4e71-b9a4-728011bfe8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:02.985802 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811d221-ec85-4e71-b9a4-728011bfe8aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:02.986501 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811d221-ec85-4e71-b9a4-728011bfe8aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:02.988433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811d221-ec85-4e71-b9a4-728011bfe8aa admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:03.017552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1332/2679] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:02 2022] POST /identity/v3/projects => generated 409 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:03.022655 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0ad4b-0d11-495c-b451-08010457a4ab None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:03.034514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc0ad4b-0d11-495c-b451-08010457a4ab None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:03.105961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbe162-d1a9-41c8-b2b4-1780534ea379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:03.106665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbe162-d1a9-41c8-b2b4-1780534ea379 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:03.107014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbe162-d1a9-41c8-b2b4-1780534ea379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:03.107363 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbe162-d1a9-41c8-b2b4-1780534ea379 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:03.108256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbe162-d1a9-41c8-b2b4-1780534ea379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:03.111804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbe162-d1a9-41c8-b2b4-1780534ea379 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:03.145254 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1348/2680] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:03 2022] POST /identity/v3/users => generated 311 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:03.151421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811d221-ec85-4e71-b9a4-728011bfe8aa admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:03.169256 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811d221-ec85-4e71-b9a4-728011bfe8aa admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:03.176909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaf645-4836-4f5c-abe6-eabcc9a87c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:03.178048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaf645-4836-4f5c-abe6-eabcc9a87c53 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:03.178483 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaf645-4836-4f5c-abe6-eabcc9a87c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:03.178982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddaf645-4836-4f5c-abe6-eabcc9a87c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:03.180322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddaf645-4836-4f5c-abe6-eabcc9a87c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:03.183737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddaf645-4836-4f5c-abe6-eabcc9a87c53 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:03.203415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1333/2681] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:03.208037 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbe162-d1a9-41c8-b2b4-1780534ea379 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:03.219770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbe162-d1a9-41c8-b2b4-1780534ea379 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:03.224734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7eb2d-4faa-43d5-aef1-191ea61a13ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:03.225680 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7eb2d-4faa-43d5-aef1-191ea61a13ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:03.225903 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7eb2d-4faa-43d5-aef1-191ea61a13ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:03.226204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7eb2d-4faa-43d5-aef1-191ea61a13ec admin admin] PATH_INFO: `/v3/projects/6861e2f3d00e443aae9b5df868602957/users/49316ca4eb6146369f0e13c570917194/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:03.246220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7eb2d-4faa-43d5-aef1-191ea61a13ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6861e2f3d00e443aae9b5df868602957, user_id=49316ca4eb6146369f0e13c570917194, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:03.249383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7eb2d-4faa-43d5-aef1-191ea61a13ec admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:03.267315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1349/2682] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:03 2022] PUT /identity/v3/projects/6861e2f3d00e443aae9b5df868602957/users/49316ca4eb6146369f0e13c570917194/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:03.272168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaf645-4836-4f5c-abe6-eabcc9a87c53 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:03.284986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddaf645-4836-4f5c-abe6-eabcc9a87c53 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:03.355265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e149e5c4-14ea-4fe3-9e05-29fcff9a7096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:03.356265 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149e5c4-14ea-4fe3-9e05-29fcff9a7096 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:03.356511 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149e5c4-14ea-4fe3-9e05-29fcff9a7096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:03.356885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e149e5c4-14ea-4fe3-9e05-29fcff9a7096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:03.357806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149e5c4-14ea-4fe3-9e05-29fcff9a7096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:03.359924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e149e5c4-14ea-4fe3-9e05-29fcff9a7096 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:03.373400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1334/2683] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:39:03.377898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7eb2d-4faa-43d5-aef1-191ea61a13ec admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:03.392769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7eb2d-4faa-43d5-aef1-191ea61a13ec admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:03.399181 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e7c52-592c-44c9-ae3c-eec55a440429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:03.400889 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e7c52-592c-44c9-ae3c-eec55a440429 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:03.401270 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e7c52-592c-44c9-ae3c-eec55a440429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:03.401744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e7c52-592c-44c9-ae3c-eec55a440429 admin admin] PATH_INFO: `/v3/projects/6861e2f3d00e443aae9b5df868602957/users/49316ca4eb6146369f0e13c570917194/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:03.405225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e7c52-592c-44c9-ae3c-eec55a440429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6861e2f3d00e443aae9b5df868602957, user_id=49316ca4eb6146369f0e13c570917194, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:03.408195 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e7c52-592c-44c9-ae3c-eec55a440429 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:03.420139 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1350/2684] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:03 2022] PUT /identity/v3/projects/6861e2f3d00e443aae9b5df868602957/users/49316ca4eb6146369f0e13c570917194/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:03.430953 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29f00a-9282-4c01-8fd3-45e1e84d5272 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:03.431394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29f00a-9282-4c01-8fd3-45e1e84d5272 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:03.431708 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29f00a-9282-4c01-8fd3-45e1e84d5272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:03.469438 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-be29f00a-9282-4c01-8fd3-45e1e84d5272 None None] MFA Rules not processed for user `49316ca4eb6146369f0e13c570917194`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:03.470750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-be29f00a-9282-4c01-8fd3-45e1e84d5272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:03.501482 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:03.579977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1335/2685] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:03 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:03.591416 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e7c52-592c-44c9-ae3c-eec55a440429 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:03.593266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-819e7c52-592c-44c9-ae3c-eec55a440429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:03.665217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e7c52-592c-44c9-ae3c-eec55a440429 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:03.669568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8325a84-13bc-4a29-a7e7-a6db2f8d24a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:03.670365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8325a84-13bc-4a29-a7e7-a6db2f8d24a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:03.670515 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8325a84-13bc-4a29-a7e7-a6db2f8d24a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:03.670829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8325a84-13bc-4a29-a7e7-a6db2f8d24a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:03.685288 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8325a84-13bc-4a29-a7e7-a6db2f8d24a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:03.685288 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8325a84-13bc-4a29-a7e7-a6db2f8d24a9 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:03.705839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1351/2686] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:03.747036 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29f00a-9282-4c01-8fd3-45e1e84d5272 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:03.748464 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-be29f00a-9282-4c01-8fd3-45e1e84d5272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:03.816573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29f00a-9282-4c01-8fd3-45e1e84d5272 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:03.820650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a3785-ce5e-4189-bd63-543e5e99e613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:03.821386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a3785-ce5e-4189-bd63-543e5e99e613 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:03.821678 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a3785-ce5e-4189-bd63-543e5e99e613 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:03.822562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a3785-ce5e-4189-bd63-543e5e99e613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:03.824943 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fa7a3785-ce5e-4189-bd63-543e5e99e613 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:03.914093 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8325a84-13bc-4a29-a7e7-a6db2f8d24a9 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:03.925443 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8325a84-13bc-4a29-a7e7-a6db2f8d24a9 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:03.926202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a3785-ce5e-4189-bd63-543e5e99e613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:03.927948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a3785-ce5e-4189-bd63-543e5e99e613 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:03.929434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53926a49-446b-4ab9-b8b1-8fc7559dc411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:03.930260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53926a49-446b-4ab9-b8b1-8fc7559dc411 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:03.930546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53926a49-446b-4ab9-b8b1-8fc7559dc411 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:03.930852 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53926a49-446b-4ab9-b8b1-8fc7559dc411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:03.942487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1336/2687] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:04.014147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:04.036451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53926a49-446b-4ab9-b8b1-8fc7559dc411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:04.039517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53926a49-446b-4ab9-b8b1-8fc7559dc411 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:04.055172 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1352/2688] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:39:08.154523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a3785-ce5e-4189-bd63-543e5e99e613 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:08.156130 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fa7a3785-ce5e-4189-bd63-543e5e99e613 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:08.179184 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a3785-ce5e-4189-bd63-543e5e99e613 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:08.184790 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cddad8-ceb9-4ccd-857d-ccd2089a6045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:08.185670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cddad8-ceb9-4ccd-857d-ccd2089a6045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:08.185952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cddad8-ceb9-4ccd-857d-ccd2089a6045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:08.186223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cddad8-ceb9-4ccd-857d-ccd2089a6045 admin admin] PATH_INFO: `/v3/users/e6c2070c6d2a450c887b8ca6b97c4b38` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:08.187440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cddad8-ceb9-4ccd-857d-ccd2089a6045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6c2070c6d2a450c887b8ca6b97c4b38)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:08.189403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cddad8-ceb9-4ccd-857d-ccd2089a6045 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:08.244730 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-55cddad8-ceb9-4ccd-857d-ccd2089a6045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6c2070c6d2a450c887b8ca6b97c4b38'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:08.252651 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-55cddad8-ceb9-4ccd-857d-ccd2089a6045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6c2070c6d2a450c887b8ca6b97c4b38'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:08.256760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:08.262219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-55cddad8-ceb9-4ccd-857d-ccd2089a6045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6c2070c6d2a450c887b8ca6b97c4b38'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:08.285934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-55cddad8-ceb9-4ccd-857d-ccd2089a6045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6c2070c6d2a450c887b8ca6b97c4b38'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:08.286741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-55cddad8-ceb9-4ccd-857d-ccd2089a6045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6c2070c6d2a450c887b8ca6b97c4b38'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:08.297215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1337/2689] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:08 2022] DELETE /identity/v3/users/e6c2070c6d2a450c887b8ca6b97c4b38 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:08.506050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53926a49-446b-4ab9-b8b1-8fc7559dc411 service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:08.519092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53926a49-446b-4ab9-b8b1-8fc7559dc411 service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:08.599460 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:08.600345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:08.600656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:08.600933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] PATH_INFO: `/v3/projects/60ee8039b1f4461c996165d5801b91f1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:08.602225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60ee8039b1f4461c996165d5801b91f1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:08.604779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:08.663934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60ee8039b1f4461c996165d5801b91f1', 'request_id': 'req-4c6bb08a-0e29-4205-8f46-a27c040c9343'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:08.673982 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60ee8039b1f4461c996165d5801b91f1', 'request_id': 'req-4c6bb08a-0e29-4205-8f46-a27c040c9343'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:08.681964 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60ee8039b1f4461c996165d5801b91f1', 'request_id': 'req-4c6bb08a-0e29-4205-8f46-a27c040c9343'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:08.693772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] The token cache is being invalidate because project 60ee8039b1f4461c996165d5801b91f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:08.694046 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:08.694768 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-4c6bb08a-0e29-4205-8f46-a27c040c9343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:08.696341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1353/2690] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:08 2022] DELETE /identity/v3/projects/60ee8039b1f4461c996165d5801b91f1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:08.702008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ecf5c-2cd9-42d3-817d-a87f993f62fb None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:08.702407 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ecf5c-2cd9-42d3-817d-a87f993f62fb None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:08.702801 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ecf5c-2cd9-42d3-817d-a87f993f62fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:08.727087 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-1d1ecf5c-2cd9-42d3-817d-a87f993f62fb None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:08.728672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1d1ecf5c-2cd9-42d3-817d-a87f993f62fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:08.805266 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1338/2691] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:08.816266 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26583f3-057f-4a95-931c-dad1f6fa3c5a None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:08.816559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26583f3-057f-4a95-931c-dad1f6fa3c5a None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:08.816849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26583f3-057f-4a95-931c-dad1f6fa3c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:08.839816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a26583f3-057f-4a95-931c-dad1f6fa3c5a None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:08.841303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a26583f3-057f-4a95-931c-dad1f6fa3c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:08.850886 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1354/2692] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:08.857011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ecf5c-2cd9-42d3-817d-a87f993f62fb None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:08.868259 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ecf5c-2cd9-42d3-817d-a87f993f62fb None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:08.874106 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f728e-aab8-4dfd-8e6e-32bf41346fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:08.874895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f728e-aab8-4dfd-8e6e-32bf41346fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:08.875223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f728e-aab8-4dfd-8e6e-32bf41346fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:08.875551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f728e-aab8-4dfd-8e6e-32bf41346fe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:08.876334 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f728e-aab8-4dfd-8e6e-32bf41346fe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:39:08.876705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f728e-aab8-4dfd-8e6e-32bf41346fe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:08.878515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f728e-aab8-4dfd-8e6e-32bf41346fe7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:08.893623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1339/2693] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:39:08.899276 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26583f3-057f-4a95-931c-dad1f6fa3c5a None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:08.917216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26583f3-057f-4a95-931c-dad1f6fa3c5a None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:08.922433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada7e96-6918-4e65-a780-a82906707ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:08.923156 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada7e96-6918-4e65-a780-a82906707ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:08.923440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada7e96-6918-4e65-a780-a82906707ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:08.923715 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada7e96-6918-4e65-a780-a82906707ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:08.924429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada7e96-6918-4e65-a780-a82906707ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:08.926395 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada7e96-6918-4e65-a780-a82906707ce5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:08.971398 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1355/2694] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:08 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:08.972724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f728e-aab8-4dfd-8e6e-32bf41346fe7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:08.987219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f728e-aab8-4dfd-8e6e-32bf41346fe7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:09.062127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e87b0-affa-41af-be72-f8b6fdcc97cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:09.062888 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e87b0-affa-41af-be72-f8b6fdcc97cd admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.063179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e87b0-affa-41af-be72-f8b6fdcc97cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.063462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e87b0-affa-41af-be72-f8b6fdcc97cd admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.064200 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e87b0-affa-41af-be72-f8b6fdcc97cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:09.066232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e87b0-affa-41af-be72-f8b6fdcc97cd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:09.090242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1340/2695] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:08 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:09.094617 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada7e96-6918-4e65-a780-a82906707ce5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:09.106451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada7e96-6918-4e65-a780-a82906707ce5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:09.111566 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a1207-2387-4b96-a2a9-9077c7d764b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:09.112299 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a1207-2387-4b96-a2a9-9077c7d764b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.112726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a1207-2387-4b96-a2a9-9077c7d764b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.113060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a1207-2387-4b96-a2a9-9077c7d764b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.113851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a1207-2387-4b96-a2a9-9077c7d764b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:09.115635 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a1207-2387-4b96-a2a9-9077c7d764b2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:09.129988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1356/2696] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:09.136157 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e87b0-affa-41af-be72-f8b6fdcc97cd admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:09.149876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e87b0-affa-41af-be72-f8b6fdcc97cd admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:09.155459 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa83fb38-2bae-4c65-8eef-cd05b1412c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:09.156380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83fb38-2bae-4c65-8eef-cd05b1412c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.156500 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83fb38-2bae-4c65-8eef-cd05b1412c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.156710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa83fb38-2bae-4c65-8eef-cd05b1412c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.157496 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83fb38-2bae-4c65-8eef-cd05b1412c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:09.159171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa83fb38-2bae-4c65-8eef-cd05b1412c8e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:09.173228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1341/2697] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:09.177508 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a1207-2387-4b96-a2a9-9077c7d764b2 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:09.188573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a1207-2387-4b96-a2a9-9077c7d764b2 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:09.193471 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c786c1-7afa-411e-adbd-716df9a90ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:09.194376 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c786c1-7afa-411e-adbd-716df9a90ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.194695 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c786c1-7afa-411e-adbd-716df9a90ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.195027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c786c1-7afa-411e-adbd-716df9a90ccf admin admin] PATH_INFO: `/v3/projects/329ae50ff4bc41009089c63379591a27/users/04ccdfebcfed484a86f0e13ff0e6d4d9/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.215467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c786c1-7afa-411e-adbd-716df9a90ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=329ae50ff4bc41009089c63379591a27, user_id=04ccdfebcfed484a86f0e13ff0e6d4d9, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:09.219117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c786c1-7afa-411e-adbd-716df9a90ccf admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:09.241955 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1357/2698] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:09 2022] PUT /identity/v3/projects/329ae50ff4bc41009089c63379591a27/users/04ccdfebcfed484a86f0e13ff0e6d4d9/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:09.253835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1025a38-aee7-4f08-b894-b0f825b27ead None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.254352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1025a38-aee7-4f08-b894-b0f825b27ead None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.254978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1025a38-aee7-4f08-b894-b0f825b27ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.287328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f1025a38-aee7-4f08-b894-b0f825b27ead None None] MFA Rules not processed for user `04ccdfebcfed484a86f0e13ff0e6d4d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:09.288621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f1025a38-aee7-4f08-b894-b0f825b27ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:09.328779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:09.375474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1342/2699] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:09.381121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c786c1-7afa-411e-adbd-716df9a90ccf admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:09.412478 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:09.472812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c786c1-7afa-411e-adbd-716df9a90ccf admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:09.477824 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775a0ac-a713-4ee2-9516-e270add7d7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:09.479335 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775a0ac-a713-4ee2-9516-e270add7d7b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.479637 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775a0ac-a713-4ee2-9516-e270add7d7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.479917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775a0ac-a713-4ee2-9516-e270add7d7b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.480616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775a0ac-a713-4ee2-9516-e270add7d7b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:09.482642 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775a0ac-a713-4ee2-9516-e270add7d7b7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:09.512681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1358/2700] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:09 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:09.517140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1025a38-aee7-4f08-b894-b0f825b27ead None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:09.529284 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1025a38-aee7-4f08-b894-b0f825b27ead None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:09.596568 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd964563-cafb-4602-9fdb-8f4383ec6869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:09.597261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd964563-cafb-4602-9fdb-8f4383ec6869 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.597548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd964563-cafb-4602-9fdb-8f4383ec6869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.597844 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd964563-cafb-4602-9fdb-8f4383ec6869 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.598537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd964563-cafb-4602-9fdb-8f4383ec6869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:09.600533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd964563-cafb-4602-9fdb-8f4383ec6869 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:09.625835 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1343/2701] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:09 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:09.630773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775a0ac-a713-4ee2-9516-e270add7d7b7 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:09.647107 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775a0ac-a713-4ee2-9516-e270add7d7b7 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:09.652764 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabf9cd-ed17-446c-b3b9-2d329c4812e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:09.653466 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabf9cd-ed17-446c-b3b9-2d329c4812e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.653737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabf9cd-ed17-446c-b3b9-2d329c4812e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.654017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabf9cd-ed17-446c-b3b9-2d329c4812e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.654793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabf9cd-ed17-446c-b3b9-2d329c4812e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:09.656432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cabf9cd-ed17-446c-b3b9-2d329c4812e8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:09.669353 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1359/2702] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:09.674115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd964563-cafb-4602-9fdb-8f4383ec6869 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:09.685429 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd964563-cafb-4602-9fdb-8f4383ec6869 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:09.690341 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdaa67-305a-4136-8b74-709c7917da3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:09.691285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdaa67-305a-4136-8b74-709c7917da3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.691575 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdaa67-305a-4136-8b74-709c7917da3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.691884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdaa67-305a-4136-8b74-709c7917da3b admin admin] PATH_INFO: `/v3/projects/2338c3e5b108446b8feb58201451d2d1/users/7a744bcbae334ad598a01d696e91791b/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.714180 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdaa67-305a-4136-8b74-709c7917da3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2338c3e5b108446b8feb58201451d2d1, user_id=7a744bcbae334ad598a01d696e91791b, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:09.718979 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdaa67-305a-4136-8b74-709c7917da3b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:09.739870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1344/2703] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:09 2022] PUT /identity/v3/projects/2338c3e5b108446b8feb58201451d2d1/users/7a744bcbae334ad598a01d696e91791b/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:09.745540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabf9cd-ed17-446c-b3b9-2d329c4812e8 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:09.757168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cabf9cd-ed17-446c-b3b9-2d329c4812e8 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:09.827926 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbac849-0d15-481f-aac1-da4b37377546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:09.829040 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbac849-0d15-481f-aac1-da4b37377546 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.829653 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbac849-0d15-481f-aac1-da4b37377546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.830120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbac849-0d15-481f-aac1-da4b37377546 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.831879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbac849-0d15-481f-aac1-da4b37377546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:09.834957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbac849-0d15-481f-aac1-da4b37377546 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:09.854265 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1360/2704] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:09.860800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdaa67-305a-4136-8b74-709c7917da3b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:09.876164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdaa67-305a-4136-8b74-709c7917da3b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:09.883757 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cc63c-fc16-410d-9a77-72ec0edff545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:09.885836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cc63c-fc16-410d-9a77-72ec0edff545 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.885836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cc63c-fc16-410d-9a77-72ec0edff545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.888361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514cc63c-fc16-410d-9a77-72ec0edff545 admin admin] PATH_INFO: `/v3/projects/2338c3e5b108446b8feb58201451d2d1/users/7a744bcbae334ad598a01d696e91791b/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.891766 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cc63c-fc16-410d-9a77-72ec0edff545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2338c3e5b108446b8feb58201451d2d1, user_id=7a744bcbae334ad598a01d696e91791b, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:09.895680 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514cc63c-fc16-410d-9a77-72ec0edff545 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:09.916205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1345/2705] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:09 2022] PUT /identity/v3/projects/2338c3e5b108446b8feb58201451d2d1/users/7a744bcbae334ad598a01d696e91791b/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:09.928212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc197b6c-5192-4b3a-b78c-0cb33f85056b None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:09.928529 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc197b6c-5192-4b3a-b78c-0cb33f85056b None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:09.928885 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc197b6c-5192-4b3a-b78c-0cb33f85056b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:09.976499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bc197b6c-5192-4b3a-b78c-0cb33f85056b None None] MFA Rules not processed for user `7a744bcbae334ad598a01d696e91791b`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:09.978891 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bc197b6c-5192-4b3a-b78c-0cb33f85056b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:10.081706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1361/2706] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:10.095071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cc63c-fc16-410d-9a77-72ec0edff545 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:10.096936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-514cc63c-fc16-410d-9a77-72ec0edff545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:10.189085 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514cc63c-fc16-410d-9a77-72ec0edff545 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:10.196799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4f1cb1-019d-4393-bb45-75131a39fdd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:10.198640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f1cb1-019d-4393-bb45-75131a39fdd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:10.199269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f1cb1-019d-4393-bb45-75131a39fdd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:10.200034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4f1cb1-019d-4393-bb45-75131a39fdd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:10.222189 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f1cb1-019d-4393-bb45-75131a39fdd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:10.225132 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4f1cb1-019d-4393-bb45-75131a39fdd6 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:10.240839 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1346/2707] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:10 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:10.725448 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc197b6c-5192-4b3a-b78c-0cb33f85056b None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:10.742052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:10.802322 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc197b6c-5192-4b3a-b78c-0cb33f85056b None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:10.807937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:10.808872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:10.809135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:10.809428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] PATH_INFO: `/v3/users/04ccdfebcfed484a86f0e13ff0e6d4d9` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:10.810831 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04ccdfebcfed484a86f0e13ff0e6d4d9)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:10.812778 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:10.869022 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04ccdfebcfed484a86f0e13ff0e6d4d9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:10.882446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04ccdfebcfed484a86f0e13ff0e6d4d9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:10.883205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04ccdfebcfed484a86f0e13ff0e6d4d9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:10.898061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04ccdfebcfed484a86f0e13ff0e6d4d9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:10.907216 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04ccdfebcfed484a86f0e13ff0e6d4d9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:10.914047 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1362/2708] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:10 2022] DELETE /identity/v3/users/04ccdfebcfed484a86f0e13ff0e6d4d9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:10.920514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f010f45-f24b-490a-b928-77488d86e940 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:10.920905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f010f45-f24b-490a-b928-77488d86e940 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:10.921352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f010f45-f24b-490a-b928-77488d86e940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:10.943577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3f010f45-f24b-490a-b928-77488d86e940 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:10.945874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3f010f45-f24b-490a-b928-77488d86e940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:10.973393 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:11.032023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1347/2709] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:11.044920 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:11.046477 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:11.118564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a5ea8-1129-4197-9c8d-c348f09b8fd9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:11.123553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24479e76-9acb-416b-9085-74ff7806b6a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:11.124236 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24479e76-9acb-416b-9085-74ff7806b6a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:11.124525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24479e76-9acb-416b-9085-74ff7806b6a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:11.124805 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24479e76-9acb-416b-9085-74ff7806b6a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:11.136394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24479e76-9acb-416b-9085-74ff7806b6a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:11.137953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24479e76-9acb-416b-9085-74ff7806b6a2 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:11.153228 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1363/2710] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:11.489286 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f010f45-f24b-490a-b928-77488d86e940 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:11.491501 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3f010f45-f24b-490a-b928-77488d86e940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:11.507012 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f010f45-f24b-490a-b928-77488d86e940 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:11.515126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:11.516047 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:11.516330 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:11.516609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] PATH_INFO: `/v3/projects/329ae50ff4bc41009089c63379591a27` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:11.517939 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] RBAC: Authorizing `identity:delete_project(project_id=329ae50ff4bc41009089c63379591a27)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:11.520037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:11.583933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329ae50ff4bc41009089c63379591a27', 'request_id': 'req-4bf52a5a-9246-4406-9347-76ae91f7f1af'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:11.594697 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329ae50ff4bc41009089c63379591a27', 'request_id': 'req-4bf52a5a-9246-4406-9347-76ae91f7f1af'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:11.603217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329ae50ff4bc41009089c63379591a27', 'request_id': 'req-4bf52a5a-9246-4406-9347-76ae91f7f1af'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:11.621506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] The token cache is being invalidate because project 329ae50ff4bc41009089c63379591a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:11.622008 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:11.622885 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:11.624492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1348/2711] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:11 2022] DELETE /identity/v3/projects/329ae50ff4bc41009089c63379591a27 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:11.628582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24479e76-9acb-416b-9085-74ff7806b6a2 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:11.640190 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24479e76-9acb-416b-9085-74ff7806b6a2 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:11.694730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:11.741533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded767d6-e8a8-4687-ba44-737ee271c13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:11.742436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded767d6-e8a8-4687-ba44-737ee271c13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:11.742773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded767d6-e8a8-4687-ba44-737ee271c13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:11.743052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded767d6-e8a8-4687-ba44-737ee271c13b admin admin] PATH_INFO: `/v3/users/7a744bcbae334ad598a01d696e91791b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:11.744954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded767d6-e8a8-4687-ba44-737ee271c13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a744bcbae334ad598a01d696e91791b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:11.746988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded767d6-e8a8-4687-ba44-737ee271c13b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:11.809645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ded767d6-e8a8-4687-ba44-737ee271c13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a744bcbae334ad598a01d696e91791b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:11.834234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ded767d6-e8a8-4687-ba44-737ee271c13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a744bcbae334ad598a01d696e91791b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:11.835030 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ded767d6-e8a8-4687-ba44-737ee271c13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a744bcbae334ad598a01d696e91791b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:11.845153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ded767d6-e8a8-4687-ba44-737ee271c13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a744bcbae334ad598a01d696e91791b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:11.855298 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ded767d6-e8a8-4687-ba44-737ee271c13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a744bcbae334ad598a01d696e91791b'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:11.862404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1364/2712] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:11 2022] DELETE /identity/v3/users/7a744bcbae334ad598a01d696e91791b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:12.206972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:12.209342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:12.232795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf52a5a-9246-4406-9347-76ae91f7f1af admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:12.303386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:12.304261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:12.304544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:12.304818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] PATH_INFO: `/v3/projects/2338c3e5b108446b8feb58201451d2d1` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:12.306115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2338c3e5b108446b8feb58201451d2d1)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:12.308736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:12.377882 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2338c3e5b108446b8feb58201451d2d1', 'request_id': 'req-2ff9b1e2-159a-4653-97d1-9dec7551a16a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:12.387228 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:12.391178 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2338c3e5b108446b8feb58201451d2d1', 'request_id': 'req-2ff9b1e2-159a-4653-97d1-9dec7551a16a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:12.401474 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2338c3e5b108446b8feb58201451d2d1', 'request_id': 'req-2ff9b1e2-159a-4653-97d1-9dec7551a16a'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:12.417250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] The token cache is being invalidate because project 2338c3e5b108446b8feb58201451d2d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:12.417682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:12.418459 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-2ff9b1e2-159a-4653-97d1-9dec7551a16a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:12.420503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1349/2713] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:12 2022] DELETE /identity/v3/projects/2338c3e5b108446b8feb58201451d2d1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:12.427302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8f005-74d8-4086-b3e8-75b5aee69228 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:12.427664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8f005-74d8-4086-b3e8-75b5aee69228 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:12.428051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8f005-74d8-4086-b3e8-75b5aee69228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:12.450291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6be8f005-74d8-4086-b3e8-75b5aee69228 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:12.452315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6be8f005-74d8-4086-b3e8-75b5aee69228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:12.572365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1365/2714] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:12.582188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f28d6-c9ff-4129-b9aa-d1c594acd0f0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:12.583336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f28d6-c9ff-4129-b9aa-d1c594acd0f0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:12.583336 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f28d6-c9ff-4129-b9aa-d1c594acd0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:12.604044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-d83f28d6-c9ff-4129-b9aa-d1c594acd0f0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:12.605328 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d83f28d6-c9ff-4129-b9aa-d1c594acd0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:12.622980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1350/2715] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:12.625802 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8f005-74d8-4086-b3e8-75b5aee69228 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:12.648947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8f005-74d8-4086-b3e8-75b5aee69228 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:12.663961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff660e5-b957-4f3a-a8d2-03bbe0000b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:12.665991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff660e5-b957-4f3a-a8d2-03bbe0000b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:12.667161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff660e5-b957-4f3a-a8d2-03bbe0000b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:12.667937 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff660e5-b957-4f3a-a8d2-03bbe0000b8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:12.669744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff660e5-b957-4f3a-a8d2-03bbe0000b8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:39:12.670484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff660e5-b957-4f3a-a8d2-03bbe0000b8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:12.674194 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff660e5-b957-4f3a-a8d2-03bbe0000b8f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:12.708836 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1366/2716] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:39:12.710887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f28d6-c9ff-4129-b9aa-d1c594acd0f0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:12.724862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f28d6-c9ff-4129-b9aa-d1c594acd0f0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:12.731374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bac496-84c1-46ef-bec2-1f699b793051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:12.732217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bac496-84c1-46ef-bec2-1f699b793051 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:12.732659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bac496-84c1-46ef-bec2-1f699b793051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:12.733044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bac496-84c1-46ef-bec2-1f699b793051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:12.733982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bac496-84c1-46ef-bec2-1f699b793051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:12.736279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bac496-84c1-46ef-bec2-1f699b793051 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:12.778405 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1351/2717] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:12 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:12.782532 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff660e5-b957-4f3a-a8d2-03bbe0000b8f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:12.795189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff660e5-b957-4f3a-a8d2-03bbe0000b8f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:12.848575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:12.880774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2486de4-1d0e-46bc-b9a4-1ffeb71110f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:12.881752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2486de4-1d0e-46bc-b9a4-1ffeb71110f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:12.882285 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2486de4-1d0e-46bc-b9a4-1ffeb71110f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:12.882892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2486de4-1d0e-46bc-b9a4-1ffeb71110f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:12.883819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2486de4-1d0e-46bc-b9a4-1ffeb71110f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:12.886177 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2486de4-1d0e-46bc-b9a4-1ffeb71110f7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:12.935447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1367/2718] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:12 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:12.940875 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bac496-84c1-46ef-bec2-1f699b793051 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:12.953644 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bac496-84c1-46ef-bec2-1f699b793051 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:12.959251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3597-c19c-436b-be99-b2acdd7b074f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:12.960059 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3597-c19c-436b-be99-b2acdd7b074f admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:12.960343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3597-c19c-436b-be99-b2acdd7b074f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:12.960638 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb3597-c19c-436b-be99-b2acdd7b074f admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:12.961473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb3597-c19c-436b-be99-b2acdd7b074f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:12.963510 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb3597-c19c-436b-be99-b2acdd7b074f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:12.977761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1352/2719] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:39:12.984086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2486de4-1d0e-46bc-b9a4-1ffeb71110f7 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:12.997510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2486de4-1d0e-46bc-b9a4-1ffeb71110f7 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:13.003693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be5f56-81ee-455c-9463-ed9f71c06f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:13.004729 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be5f56-81ee-455c-9463-ed9f71c06f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:13.005225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be5f56-81ee-455c-9463-ed9f71c06f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:13.005641 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be5f56-81ee-455c-9463-ed9f71c06f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:13.006659 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be5f56-81ee-455c-9463-ed9f71c06f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:13.008770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be5f56-81ee-455c-9463-ed9f71c06f58 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:13.029976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1368/2720] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:13.032240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3597-c19c-436b-be99-b2acdd7b074f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:13.045358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb3597-c19c-436b-be99-b2acdd7b074f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:13.051325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fdd29-f3d0-49c2-8bf6-48de1ce05711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:13.052325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fdd29-f3d0-49c2-8bf6-48de1ce05711 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:13.052617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fdd29-f3d0-49c2-8bf6-48de1ce05711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:13.052983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fdd29-f3d0-49c2-8bf6-48de1ce05711 admin admin] PATH_INFO: `/v3/projects/f7ac3837102744288712238cd6feb4de/users/9f42a2fbff2540d9ab7c9ef07e3c3881/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:13.073242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fdd29-f3d0-49c2-8bf6-48de1ce05711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ac3837102744288712238cd6feb4de, user_id=9f42a2fbff2540d9ab7c9ef07e3c3881, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:13.076640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fdd29-f3d0-49c2-8bf6-48de1ce05711 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:13.094050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1353/2721] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:13 2022] PUT /identity/v3/projects/f7ac3837102744288712238cd6feb4de/users/9f42a2fbff2540d9ab7c9ef07e3c3881/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:13.106958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28252d-b21b-460e-ba1a-690eb255fc06 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:13.107464 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28252d-b21b-460e-ba1a-690eb255fc06 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:13.108013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28252d-b21b-460e-ba1a-690eb255fc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:13.144202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-ff28252d-b21b-460e-ba1a-690eb255fc06 None None] MFA Rules not processed for user `9f42a2fbff2540d9ab7c9ef07e3c3881`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:13.145800 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ff28252d-b21b-460e-ba1a-690eb255fc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:13.236488 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1369/2722] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:13.241689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fdd29-f3d0-49c2-8bf6-48de1ce05711 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:13.314109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fdd29-f3d0-49c2-8bf6-48de1ce05711 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:13.321729 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9dcd19-b1ca-43b1-9f87-ada14a947e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:13.324301 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dcd19-b1ca-43b1-9f87-ada14a947e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:13.324668 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dcd19-b1ca-43b1-9f87-ada14a947e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:13.324969 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9dcd19-b1ca-43b1-9f87-ada14a947e7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:13.325805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9dcd19-b1ca-43b1-9f87-ada14a947e7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:13.328324 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9dcd19-b1ca-43b1-9f87-ada14a947e7e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:13.360702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1354/2723] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:13 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:13.365743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28252d-b21b-460e-ba1a-690eb255fc06 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:13.382827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28252d-b21b-460e-ba1a-690eb255fc06 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:13.468648 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99a46a-fd89-4b85-878c-75203980c598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:13.469377 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99a46a-fd89-4b85-878c-75203980c598 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:13.469683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99a46a-fd89-4b85-878c-75203980c598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:13.469949 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99a46a-fd89-4b85-878c-75203980c598 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:13.470723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99a46a-fd89-4b85-878c-75203980c598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:13.472703 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99a46a-fd89-4b85-878c-75203980c598 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:13.498957 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1370/2724] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:13 2022] POST /identity/v3/users => generated 300 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:13.505258 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9dcd19-b1ca-43b1-9f87-ada14a947e7e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:13.524610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9dcd19-b1ca-43b1-9f87-ada14a947e7e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:13.533158 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c84b51-f4f0-4f21-b15d-813292d17563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:13.533894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c84b51-f4f0-4f21-b15d-813292d17563 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:13.534216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c84b51-f4f0-4f21-b15d-813292d17563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:13.534499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c84b51-f4f0-4f21-b15d-813292d17563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:13.535614 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c84b51-f4f0-4f21-b15d-813292d17563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:13.538406 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c84b51-f4f0-4f21-b15d-813292d17563 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:13.551452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1355/2725] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:13.555801 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99a46a-fd89-4b85-878c-75203980c598 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:13.566627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99a46a-fd89-4b85-878c-75203980c598 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:13.571661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e72dec-14f5-454f-8091-d3563efb3ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:13.573135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e72dec-14f5-454f-8091-d3563efb3ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:13.573472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e72dec-14f5-454f-8091-d3563efb3ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:13.573720 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e72dec-14f5-454f-8091-d3563efb3ba9 admin admin] PATH_INFO: `/v3/projects/c8b6ca189ac64220ac248be3a011e9f2/users/9f50a9da4b7b49da9a76290539e148f6/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:13.594858 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e72dec-14f5-454f-8091-d3563efb3ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8b6ca189ac64220ac248be3a011e9f2, user_id=9f50a9da4b7b49da9a76290539e148f6, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:13.597960 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e72dec-14f5-454f-8091-d3563efb3ba9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:13.613494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1371/2726] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:13 2022] PUT /identity/v3/projects/c8b6ca189ac64220ac248be3a011e9f2/users/9f50a9da4b7b49da9a76290539e148f6/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:13.617647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c84b51-f4f0-4f21-b15d-813292d17563 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:13.631930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c84b51-f4f0-4f21-b15d-813292d17563 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:13.697938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:13.713735 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33d7c8-7696-415f-b5b3-ffac7c377ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:13.715034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33d7c8-7696-415f-b5b3-ffac7c377ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:13.715793 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33d7c8-7696-415f-b5b3-ffac7c377ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:13.716225 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33d7c8-7696-415f-b5b3-ffac7c377ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:13.717605 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33d7c8-7696-415f-b5b3-ffac7c377ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:13.719702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33d7c8-7696-415f-b5b3-ffac7c377ad4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:13.736671 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1356/2727] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:13.741447 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e72dec-14f5-454f-8091-d3563efb3ba9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:13.753533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e72dec-14f5-454f-8091-d3563efb3ba9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:13.758535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab54822-2811-46d9-9b9c-f27868ff9b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:13.760073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54822-2811-46d9-9b9c-f27868ff9b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:13.760357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54822-2811-46d9-9b9c-f27868ff9b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:13.760639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab54822-2811-46d9-9b9c-f27868ff9b10 admin admin] PATH_INFO: `/v3/projects/c8b6ca189ac64220ac248be3a011e9f2/users/9f50a9da4b7b49da9a76290539e148f6/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:13.763099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab54822-2811-46d9-9b9c-f27868ff9b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8b6ca189ac64220ac248be3a011e9f2, user_id=9f50a9da4b7b49da9a76290539e148f6, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:13.766152 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab54822-2811-46d9-9b9c-f27868ff9b10 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:13.773407 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:13.780305 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1372/2728] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:13 2022] PUT /identity/v3/projects/c8b6ca189ac64220ac248be3a011e9f2/users/9f50a9da4b7b49da9a76290539e148f6/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:13.790880 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d03af5-128e-42c1-bff8-3d6cbf9cf7cc None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:13.791172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d03af5-128e-42c1-bff8-3d6cbf9cf7cc None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:13.791452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d03af5-128e-42c1-bff8-3d6cbf9cf7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:13.821596 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e6d03af5-128e-42c1-bff8-3d6cbf9cf7cc None None] MFA Rules not processed for user `9f50a9da4b7b49da9a76290539e148f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:13.822924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e6d03af5-128e-42c1-bff8-3d6cbf9cf7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:13.901975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1357/2729] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:13.916212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab54822-2811-46d9-9b9c-f27868ff9b10 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:13.918616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7ab54822-2811-46d9-9b9c-f27868ff9b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:13.984829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab54822-2811-46d9-9b9c-f27868ff9b10 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:13.988725 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f8e8b-343a-489a-b67e-01938a9a5ec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:13.989427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f8e8b-343a-489a-b67e-01938a9a5ec1 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:13.989701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f8e8b-343a-489a-b67e-01938a9a5ec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:13.989975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f8e8b-343a-489a-b67e-01938a9a5ec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:14.004427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f8e8b-343a-489a-b67e-01938a9a5ec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:14.005915 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f8e8b-343a-489a-b67e-01938a9a5ec1 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:14.017539 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1373/2730] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:14.094013 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d03af5-128e-42c1-bff8-3d6cbf9cf7cc None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:14.189109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d03af5-128e-42c1-bff8-3d6cbf9cf7cc None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:14.193718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:14.194623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:14.194913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:14.195218 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] PATH_INFO: `/v3/users/9f42a2fbff2540d9ab7c9ef07e3c3881` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:14.196491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f42a2fbff2540d9ab7c9ef07e3c3881)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:14.199119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:14.269622 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f42a2fbff2540d9ab7c9ef07e3c3881'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:14.279894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f42a2fbff2540d9ab7c9ef07e3c3881'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:14.290850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f42a2fbff2540d9ab7c9ef07e3c3881'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:14.307893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:14.311724 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f42a2fbff2540d9ab7c9ef07e3c3881'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:14.312607 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f42a2fbff2540d9ab7c9ef07e3c3881'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:14.320427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1358/2731] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:14 2022] DELETE /identity/v3/users/9f42a2fbff2540d9ab7c9ef07e3c3881 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:14.325775 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520b299-1b1c-4114-8f65-ab4bf5b1fab0 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:14.326217 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520b299-1b1c-4114-8f65-ab4bf5b1fab0 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:14.326457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d520b299-1b1c-4114-8f65-ab4bf5b1fab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:14.351881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d520b299-1b1c-4114-8f65-ab4bf5b1fab0 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:14.353875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d520b299-1b1c-4114-8f65-ab4bf5b1fab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:14.431374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1374/2732] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:14.443223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:14.444764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:14.525478 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab642ed-c180-44b8-9b5d-803a40492eb9 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:14.529946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487deea4-0e7a-4f58-9816-e73d982578b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:14.530877 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487deea4-0e7a-4f58-9816-e73d982578b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:14.531325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487deea4-0e7a-4f58-9816-e73d982578b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:14.531810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487deea4-0e7a-4f58-9816-e73d982578b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:14.549364 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487deea4-0e7a-4f58-9816-e73d982578b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:14.552019 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487deea4-0e7a-4f58-9816-e73d982578b2 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:14.566470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1359/2733] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:14.885245 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520b299-1b1c-4114-8f65-ab4bf5b1fab0 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:14.886743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d520b299-1b1c-4114-8f65-ab4bf5b1fab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:14.899522 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d520b299-1b1c-4114-8f65-ab4bf5b1fab0 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:14.904917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:14.905789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:14.906068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:14.906335 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] PATH_INFO: `/v3/projects/f7ac3837102744288712238cd6feb4de` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:14.907583 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7ac3837102744288712238cd6feb4de)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:14.909507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:14.977274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7ac3837102744288712238cd6feb4de', 'request_id': 'req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:14.987573 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7ac3837102744288712238cd6feb4de', 'request_id': 'req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:14.996457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7ac3837102744288712238cd6feb4de', 'request_id': 'req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.009991 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] The token cache is being invalidate because project f7ac3837102744288712238cd6feb4de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:15.010349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.010898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.013013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1375/2734] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:14 2022] DELETE /identity/v3/projects/f7ac3837102744288712238cd6feb4de => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:15.017598 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487deea4-0e7a-4f58-9816-e73d982578b2 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:15.038092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487deea4-0e7a-4f58-9816-e73d982578b2 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:15.116440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80259f4-d7e5-44c4-9334-d6c081dd34d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:15.118024 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80259f4-d7e5-44c4-9334-d6c081dd34d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:15.118550 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80259f4-d7e5-44c4-9334-d6c081dd34d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:15.119126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80259f4-d7e5-44c4-9334-d6c081dd34d8 admin admin] PATH_INFO: `/v3/users/9f50a9da4b7b49da9a76290539e148f6` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:15.121371 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80259f4-d7e5-44c4-9334-d6c081dd34d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f50a9da4b7b49da9a76290539e148f6)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:15.124857 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80259f4-d7e5-44c4-9334-d6c081dd34d8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:15.198919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d80259f4-d7e5-44c4-9334-d6c081dd34d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f50a9da4b7b49da9a76290539e148f6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.211646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d80259f4-d7e5-44c4-9334-d6c081dd34d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f50a9da4b7b49da9a76290539e148f6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.223184 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d80259f4-d7e5-44c4-9334-d6c081dd34d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f50a9da4b7b49da9a76290539e148f6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.238763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d80259f4-d7e5-44c4-9334-d6c081dd34d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f50a9da4b7b49da9a76290539e148f6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.239568 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d80259f4-d7e5-44c4-9334-d6c081dd34d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f50a9da4b7b49da9a76290539e148f6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.246853 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1360/2735] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:15 2022] DELETE /identity/v3/users/9f50a9da4b7b49da9a76290539e148f6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:15.607665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:15.609128 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:15.623499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2405b-ac87-47a4-8d54-90c4dbe548a7 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:15.653998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:15.693118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:15.694014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:15.694296 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:15.694568 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] PATH_INFO: `/v3/projects/c8b6ca189ac64220ac248be3a011e9f2` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:15.695815 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8b6ca189ac64220ac248be3a011e9f2)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:15.698774 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:15.754627 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8b6ca189ac64220ac248be3a011e9f2', 'request_id': 'req-eea166f8-3fe2-4ff4-8ee3-0de049004be9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.763233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8b6ca189ac64220ac248be3a011e9f2', 'request_id': 'req-eea166f8-3fe2-4ff4-8ee3-0de049004be9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.770071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8b6ca189ac64220ac248be3a011e9f2', 'request_id': 'req-eea166f8-3fe2-4ff4-8ee3-0de049004be9'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.781777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] The token cache is being invalidate because project c8b6ca189ac64220ac248be3a011e9f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:15.782062 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.782750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-eea166f8-3fe2-4ff4-8ee3-0de049004be9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:15.784332 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1376/2736] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:15 2022] DELETE /identity/v3/projects/c8b6ca189ac64220ac248be3a011e9f2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:15.789679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737831-d038-4401-b59d-69df5546110c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:15.789679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737831-d038-4401-b59d-69df5546110c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:15.789909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737831-d038-4401-b59d-69df5546110c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:15.808849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a2737831-d038-4401-b59d-69df5546110c None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:15.810083 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a2737831-d038-4401-b59d-69df5546110c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:15.819800 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:15.873108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1361/2737] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:15.880943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3aa5aa-c67a-47fa-b436-d08966b0fca4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:15.881238 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3aa5aa-c67a-47fa-b436-d08966b0fca4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:15.881528 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3aa5aa-c67a-47fa-b436-d08966b0fca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:15.902297 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-9a3aa5aa-c67a-47fa-b436-d08966b0fca4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:15.903551 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-9a3aa5aa-c67a-47fa-b436-d08966b0fca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:15.915025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1377/2738] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:15.919639 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2737831-d038-4401-b59d-69df5546110c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:15.929707 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2737831-d038-4401-b59d-69df5546110c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:15.934528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f08ae-c460-43fa-b673-e05d1ccbb026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:15.935223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f08ae-c460-43fa-b673-e05d1ccbb026 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:15.935513 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f08ae-c460-43fa-b673-e05d1ccbb026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:15.935795 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f08ae-c460-43fa-b673-e05d1ccbb026 admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:15.936521 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f08ae-c460-43fa-b673-e05d1ccbb026 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:39:15.936833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f08ae-c460-43fa-b673-e05d1ccbb026 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:15.938433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f08ae-c460-43fa-b673-e05d1ccbb026 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:15.951095 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1362/2739] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:39:15.955456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3aa5aa-c67a-47fa-b436-d08966b0fca4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:15.965643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3aa5aa-c67a-47fa-b436-d08966b0fca4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:15.971634 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447db8a-fbad-4977-9cb6-34f50cccbb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:15.972413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447db8a-fbad-4977-9cb6-34f50cccbb8b admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:15.972708 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447db8a-fbad-4977-9cb6-34f50cccbb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:15.972993 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447db8a-fbad-4977-9cb6-34f50cccbb8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:15.973704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447db8a-fbad-4977-9cb6-34f50cccbb8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:15.975697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447db8a-fbad-4977-9cb6-34f50cccbb8b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:16.004981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1378/2740] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:15 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:16.006809 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f08ae-c460-43fa-b673-e05d1ccbb026 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:16.018021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f08ae-c460-43fa-b673-e05d1ccbb026 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:16.079934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf1b3b-76a5-472a-8235-9baf3acc4e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:16.080611 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf1b3b-76a5-472a-8235-9baf3acc4e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.080899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf1b3b-76a5-472a-8235-9baf3acc4e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.081204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf1b3b-76a5-472a-8235-9baf3acc4e8b admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.081916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf1b3b-76a5-472a-8235-9baf3acc4e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:16.083914 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf1b3b-76a5-472a-8235-9baf3acc4e8b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:16.104346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1363/2741] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:16 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:16.108589 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447db8a-fbad-4977-9cb6-34f50cccbb8b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:16.121549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447db8a-fbad-4977-9cb6-34f50cccbb8b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:16.126864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e708be-497d-4ce2-a60c-9a19aad6977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:16.127537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e708be-497d-4ce2-a60c-9a19aad6977a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.127828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e708be-497d-4ce2-a60c-9a19aad6977a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.128161 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e708be-497d-4ce2-a60c-9a19aad6977a admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.128929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e708be-497d-4ce2-a60c-9a19aad6977a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:16.131008 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e708be-497d-4ce2-a60c-9a19aad6977a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:16.143893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1379/2742] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:16.147926 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf1b3b-76a5-472a-8235-9baf3acc4e8b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:16.158517 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf1b3b-76a5-472a-8235-9baf3acc4e8b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:16.163184 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2b920-ef4e-49fd-90f2-fef30d5d5158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:16.163849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2b920-ef4e-49fd-90f2-fef30d5d5158 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.164139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2b920-ef4e-49fd-90f2-fef30d5d5158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.164420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2b920-ef4e-49fd-90f2-fef30d5d5158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.165144 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2b920-ef4e-49fd-90f2-fef30d5d5158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:16.166833 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2b920-ef4e-49fd-90f2-fef30d5d5158 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:16.177559 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1364/2743] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:16.182209 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e708be-497d-4ce2-a60c-9a19aad6977a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:16.192945 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e708be-497d-4ce2-a60c-9a19aad6977a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:16.198300 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3b166-94e2-423e-94df-59b575c51d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:16.199309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3b166-94e2-423e-94df-59b575c51d8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.199597 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3b166-94e2-423e-94df-59b575c51d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.199879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f3b166-94e2-423e-94df-59b575c51d8d admin admin] PATH_INFO: `/v3/projects/8ba92d37b94543a797e9fdfd813f5667/users/26261ed4544549938fd9991c39f20855/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.214753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3b166-94e2-423e-94df-59b575c51d8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ba92d37b94543a797e9fdfd813f5667, user_id=26261ed4544549938fd9991c39f20855, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:16.217428 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f3b166-94e2-423e-94df-59b575c51d8d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:16.230527 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1380/2744] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:16 2022] PUT /identity/v3/projects/8ba92d37b94543a797e9fdfd813f5667/users/26261ed4544549938fd9991c39f20855/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:16.240734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0d1bf-8150-45ff-a997-929ef427edbf None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.241028 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0d1bf-8150-45ff-a997-929ef427edbf None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.241311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0d1bf-8150-45ff-a997-929ef427edbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.269781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-0eb0d1bf-8150-45ff-a997-929ef427edbf None None] MFA Rules not processed for user `26261ed4544549938fd9991c39f20855`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:16.270983 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-0eb0d1bf-8150-45ff-a997-929ef427edbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:16.338166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1365/2745] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:16.342616 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3b166-94e2-423e-94df-59b575c51d8d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:16.399989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:16.413049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f3b166-94e2-423e-94df-59b575c51d8d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:16.419827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a8a18-0144-4673-a8b2-178944a0e21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:16.420491 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a8a18-0144-4673-a8b2-178944a0e21b admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.420776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a8a18-0144-4673-a8b2-178944a0e21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.421055 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a8a18-0144-4673-a8b2-178944a0e21b admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.421748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a8a18-0144-4673-a8b2-178944a0e21b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:16.423908 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a8a18-0144-4673-a8b2-178944a0e21b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:16.457754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1381/2746] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:16 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:16.462097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0d1bf-8150-45ff-a997-929ef427edbf None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:16.473492 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0d1bf-8150-45ff-a997-929ef427edbf None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:16.482315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:16.545246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3a4e1-9400-4a26-8b2d-ec19bd3f4974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:16.545936 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3a4e1-9400-4a26-8b2d-ec19bd3f4974 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.546227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3a4e1-9400-4a26-8b2d-ec19bd3f4974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.546506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3a4e1-9400-4a26-8b2d-ec19bd3f4974 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.547244 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3a4e1-9400-4a26-8b2d-ec19bd3f4974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:16.549181 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec3a4e1-9400-4a26-8b2d-ec19bd3f4974 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:16.575079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1366/2747] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:16 2022] POST /identity/v3/users => generated 300 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:16.579494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a8a18-0144-4673-a8b2-178944a0e21b admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:16.590854 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a8a18-0144-4673-a8b2-178944a0e21b admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:16.595578 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf159d7-519c-4ea1-b850-eb00613014eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:16.596256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf159d7-519c-4ea1-b850-eb00613014eb admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.596543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf159d7-519c-4ea1-b850-eb00613014eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.596821 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf159d7-519c-4ea1-b850-eb00613014eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.597553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf159d7-519c-4ea1-b850-eb00613014eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:16.599224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf159d7-519c-4ea1-b850-eb00613014eb admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:16.611256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1382/2748] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:16.616108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3a4e1-9400-4a26-8b2d-ec19bd3f4974 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:16.627151 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3a4e1-9400-4a26-8b2d-ec19bd3f4974 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:16.632250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0247d0-ccba-4aaa-aa1a-33556cbbf0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:16.633206 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0247d0-ccba-4aaa-aa1a-33556cbbf0bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.633499 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0247d0-ccba-4aaa-aa1a-33556cbbf0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.633777 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0247d0-ccba-4aaa-aa1a-33556cbbf0bd admin admin] PATH_INFO: `/v3/projects/08f3f4d957da411e8408b67330e48d16/users/e0ba46b639b24519b11a4f231ceaeaba/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.651535 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0247d0-ccba-4aaa-aa1a-33556cbbf0bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f3f4d957da411e8408b67330e48d16, user_id=e0ba46b639b24519b11a4f231ceaeaba, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:16.655122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0247d0-ccba-4aaa-aa1a-33556cbbf0bd admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:16.669874 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1367/2749] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:16 2022] PUT /identity/v3/projects/08f3f4d957da411e8408b67330e48d16/users/e0ba46b639b24519b11a4f231ceaeaba/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:16.675362 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf159d7-519c-4ea1-b850-eb00613014eb admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:16.686751 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf159d7-519c-4ea1-b850-eb00613014eb admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:16.747606 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545ebeb-4229-49dd-b081-18e0b40b312a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:16.748511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545ebeb-4229-49dd-b081-18e0b40b312a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.748628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545ebeb-4229-49dd-b081-18e0b40b312a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.748955 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545ebeb-4229-49dd-b081-18e0b40b312a admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.749717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545ebeb-4229-49dd-b081-18e0b40b312a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:16.751359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545ebeb-4229-49dd-b081-18e0b40b312a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:16.763817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1383/2750] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:39:16.768603 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0247d0-ccba-4aaa-aa1a-33556cbbf0bd admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:16.781164 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0247d0-ccba-4aaa-aa1a-33556cbbf0bd admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:16.786240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff24ef-4d9e-4617-b1e2-bbfc07de07c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:16.787239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff24ef-4d9e-4617-b1e2-bbfc07de07c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.787552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff24ef-4d9e-4617-b1e2-bbfc07de07c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.788009 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff24ef-4d9e-4617-b1e2-bbfc07de07c7 admin admin] PATH_INFO: `/v3/projects/08f3f4d957da411e8408b67330e48d16/users/e0ba46b639b24519b11a4f231ceaeaba/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.790035 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff24ef-4d9e-4617-b1e2-bbfc07de07c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f3f4d957da411e8408b67330e48d16, user_id=e0ba46b639b24519b11a4f231ceaeaba, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:16.793391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff24ef-4d9e-4617-b1e2-bbfc07de07c7 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:16.804621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1368/2751] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:16 2022] PUT /identity/v3/projects/08f3f4d957da411e8408b67330e48d16/users/e0ba46b639b24519b11a4f231ceaeaba/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:16.817730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e2cc6-fb65-4128-82f7-b7857fb8406f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:16.818289 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e2cc6-fb65-4128-82f7-b7857fb8406f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:16.818839 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e2cc6-fb65-4128-82f7-b7857fb8406f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:16.857549 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-6b1e2cc6-fb65-4128-82f7-b7857fb8406f None None] MFA Rules not processed for user `e0ba46b639b24519b11a4f231ceaeaba`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:16.858814 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6b1e2cc6-fb65-4128-82f7-b7857fb8406f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:16.942793 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1384/2752] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:16.954169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff24ef-4d9e-4617-b1e2-bbfc07de07c7 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:16.955904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e4ff24ef-4d9e-4617-b1e2-bbfc07de07c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:17.019448 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff24ef-4d9e-4617-b1e2-bbfc07de07c7 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:17.023279 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65158e-f0c3-4117-99b6-60edb4c1bca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:17.023944 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65158e-f0c3-4117-99b6-60edb4c1bca7 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:17.024731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65158e-f0c3-4117-99b6-60edb4c1bca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:17.024731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65158e-f0c3-4117-99b6-60edb4c1bca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:17.034807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65158e-f0c3-4117-99b6-60edb4c1bca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:17.036264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65158e-f0c3-4117-99b6-60edb4c1bca7 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:17.046556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1369/2753] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:17.645080 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e2cc6-fb65-4128-82f7-b7857fb8406f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:17.646934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-6b1e2cc6-fb65-4128-82f7-b7857fb8406f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:17.648188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65158e-f0c3-4117-99b6-60edb4c1bca7 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:17.708941 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:17.725534 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65158e-f0c3-4117-99b6-60edb4c1bca7 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:17.728593 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e2cc6-fb65-4128-82f7-b7857fb8406f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:17.730929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367ec51e-1b9f-4421-a965-389d4781fff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:17.731799 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ec51e-1b9f-4421-a965-389d4781fff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:17.732082 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ec51e-1b9f-4421-a965-389d4781fff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:17.732355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367ec51e-1b9f-4421-a965-389d4781fff9 admin admin] PATH_INFO: `/v3/users/26261ed4544549938fd9991c39f20855` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:17.733528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ec51e-1b9f-4421-a965-389d4781fff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26261ed4544549938fd9991c39f20855)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:17.734111 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18945f46-2e04-42d6-bc09-92004972eb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:17.735014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18945f46-2e04-42d6-bc09-92004972eb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:17.735294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18945f46-2e04-42d6-bc09-92004972eb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:17.735487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367ec51e-1b9f-4421-a965-389d4781fff9 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:17.735608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18945f46-2e04-42d6-bc09-92004972eb1b admin admin] PATH_INFO: `/v3/users/9565eb195c6044c18a70a079afb779e7` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:17.736758 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18945f46-2e04-42d6-bc09-92004972eb1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9565eb195c6044c18a70a079afb779e7)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:17.738977 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18945f46-2e04-42d6-bc09-92004972eb1b admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:17.741289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:17.787779 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-367ec51e-1b9f-4421-a965-389d4781fff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26261ed4544549938fd9991c39f20855'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:17.789850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-18945f46-2e04-42d6-bc09-92004972eb1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9565eb195c6044c18a70a079afb779e7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:17.795714 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-367ec51e-1b9f-4421-a965-389d4781fff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26261ed4544549938fd9991c39f20855'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:17.803405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-18945f46-2e04-42d6-bc09-92004972eb1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9565eb195c6044c18a70a079afb779e7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:17.804212 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-367ec51e-1b9f-4421-a965-389d4781fff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26261ed4544549938fd9991c39f20855'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:17.804356 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-18945f46-2e04-42d6-bc09-92004972eb1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9565eb195c6044c18a70a079afb779e7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:17.812174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-18945f46-2e04-42d6-bc09-92004972eb1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9565eb195c6044c18a70a079afb779e7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:17.818399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-367ec51e-1b9f-4421-a965-389d4781fff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26261ed4544549938fd9991c39f20855'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:17.819110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-367ec51e-1b9f-4421-a965-389d4781fff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26261ed4544549938fd9991c39f20855'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:17.819525 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-18945f46-2e04-42d6-bc09-92004972eb1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9565eb195c6044c18a70a079afb779e7'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:17.825346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1370/2754] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:17 2022] DELETE /identity/v3/users/26261ed4544549938fd9991c39f20855 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:17.828355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1385/2755] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:17 2022] DELETE /identity/v3/users/9565eb195c6044c18a70a079afb779e7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:17.830045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429c506-316e-4a10-9d64-5a138c1a221e None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:17.830189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af2b0-c8a8-464c-b87a-264865f11cf3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:17.830324 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429c506-316e-4a10-9d64-5a138c1a221e None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:17.830435 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af2b0-c8a8-464c-b87a-264865f11cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:17.830636 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429c506-316e-4a10-9d64-5a138c1a221e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:17.830750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af2b0-c8a8-464c-b87a-264865f11cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:17.849105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c429c506-316e-4a10-9d64-5a138c1a221e None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:17.850236 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c429c506-316e-4a10-9d64-5a138c1a221e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:17.850383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-cc9af2b0-c8a8-464c-b87a-264865f11cf3 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:17.851479 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cc9af2b0-c8a8-464c-b87a-264865f11cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:17.918320 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1371/2756] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:17.923394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1386/2757] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:17.928073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429c506-316e-4a10-9d64-5a138c1a221e None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:17.929506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c429c506-316e-4a10-9d64-5a138c1a221e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:17.933096 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9af2b0-c8a8-464c-b87a-264865f11cf3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:17.934511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cc9af2b0-c8a8-464c-b87a-264865f11cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:17.992894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429c506-316e-4a10-9d64-5a138c1a221e None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:17.996777 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9af2b0-c8a8-464c-b87a-264865f11cf3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:17.997386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608d808-2201-4b9b-b8af-9fcfc3ff49bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:17.998210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608d808-2201-4b9b-b8af-9fcfc3ff49bc service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:17.998503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608d808-2201-4b9b-b8af-9fcfc3ff49bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:17.998840 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608d808-2201-4b9b-b8af-9fcfc3ff49bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:18.000490 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08c958-942f-4fa2-8ff0-b95be6a955bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:18.001132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08c958-942f-4fa2-8ff0-b95be6a955bc service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:18.001410 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08c958-942f-4fa2-8ff0-b95be6a955bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:18.001686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc08c958-942f-4fa2-8ff0-b95be6a955bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:18.010361 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608d808-2201-4b9b-b8af-9fcfc3ff49bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:18.011618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08c958-942f-4fa2-8ff0-b95be6a955bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:18.012197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608d808-2201-4b9b-b8af-9fcfc3ff49bc service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:18.013092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc08c958-942f-4fa2-8ff0-b95be6a955bc service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:18.023429 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1387/2758] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:39:18.026948 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1372/2759] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:18.428024 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08c958-942f-4fa2-8ff0-b95be6a955bc service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:18.428211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608d808-2201-4b9b-b8af-9fcfc3ff49bc service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:18.429370 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dc08c958-942f-4fa2-8ff0-b95be6a955bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:18.429819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e608d808-2201-4b9b-b8af-9fcfc3ff49bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:18.457658 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc08c958-942f-4fa2-8ff0-b95be6a955bc service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:18.460806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608d808-2201-4b9b-b8af-9fcfc3ff49bc service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:18.463978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:18.464849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:18.465135 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:18.465472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] PATH_INFO: `/v3/projects/8ba92d37b94543a797e9fdfd813f5667` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:18.466792 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ba92d37b94543a797e9fdfd813f5667)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:18.467861 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:18.468870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:18.469222 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:18.469368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:18.469615 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] PATH_INFO: `/v3/projects/4088e3eced9742acbe941f2324d44d60` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:18.471580 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4088e3eced9742acbe941f2324d44d60)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:18.475533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:18.480021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:18.522647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:18.527036 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ba92d37b94543a797e9fdfd813f5667', 'request_id': 'req-94920479-2715-4a64-ba5b-78714a7aaf90'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.536441 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ba92d37b94543a797e9fdfd813f5667', 'request_id': 'req-94920479-2715-4a64-ba5b-78714a7aaf90'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.549987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ba92d37b94543a797e9fdfd813f5667', 'request_id': 'req-94920479-2715-4a64-ba5b-78714a7aaf90'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.553632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4088e3eced9742acbe941f2324d44d60', 'request_id': 'req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.559752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4088e3eced9742acbe941f2324d44d60', 'request_id': 'req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.566283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4088e3eced9742acbe941f2324d44d60', 'request_id': 'req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.568636 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] The token cache is being invalidate because project 8ba92d37b94543a797e9fdfd813f5667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:18.569174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.570287 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.573891 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1388/2760] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:18 2022] DELETE /identity/v3/projects/8ba92d37b94543a797e9fdfd813f5667 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:18.579730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:18.580348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] The token cache is being invalidate because project 4088e3eced9742acbe941f2324d44d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:18.580642 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.581282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.581474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:18.582862 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1373/2761] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:18 2022] DELETE /identity/v3/projects/4088e3eced9742acbe941f2324d44d60 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:18.589109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:18.591277 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:18.651732 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94920479-2715-4a64-ba5b-78714a7aaf90 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:18.669064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af11ba-f23c-42b8-91d9-f97d8ec5d51d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:18.675246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:18.676221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:18.676510 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:18.676780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] PATH_INFO: `/v3/users/49316ca4eb6146369f0e13c570917194` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:18.678191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49316ca4eb6146369f0e13c570917194)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:18.680217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:18.714327 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:18.715614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:18.715875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:18.716435 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] PATH_INFO: `/v3/users/e0ba46b639b24519b11a4f231ceaeaba` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:18.717618 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ba46b639b24519b11a4f231ceaeaba)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:18.720317 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:18.740915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49316ca4eb6146369f0e13c570917194'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.749928 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49316ca4eb6146369f0e13c570917194'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.760647 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49316ca4eb6146369f0e13c570917194'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.773906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49316ca4eb6146369f0e13c570917194'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.774674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49316ca4eb6146369f0e13c570917194'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.778920 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:18.782559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ba46b639b24519b11a4f231ceaeaba'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.783561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1374/2762] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:18 2022] DELETE /identity/v3/users/49316ca4eb6146369f0e13c570917194 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:18.801817 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ba46b639b24519b11a4f231ceaeaba'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.803000 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ba46b639b24519b11a4f231ceaeaba'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.815131 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ba46b639b24519b11a4f231ceaeaba'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.823071 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ba46b639b24519b11a4f231ceaeaba'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:18.830074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1389/2763] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:18 2022] DELETE /identity/v3/users/e0ba46b639b24519b11a4f231ceaeaba => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:19.141005 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.142888 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:19.195553 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.199543 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:19.215060 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:19.256288 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ff87d-c5ad-4d0f-81ce-52ea5f000366 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.293694 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6120b26-adfa-4895-a9c5-96786f90b873 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.298869 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.299743 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.300019 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.300459 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] PATH_INFO: `/v3/projects/08f3f4d957da411e8408b67330e48d16` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.301497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f3f4d957da411e8408b67330e48d16)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.303753 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.325781 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.326836 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.327131 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.327413 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] PATH_INFO: `/v3/projects/6861e2f3d00e443aae9b5df868602957` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.328778 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6861e2f3d00e443aae9b5df868602957)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.330792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.367855 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f3f4d957da411e8408b67330e48d16', 'request_id': 'req-90f7189b-3046-4ee3-b145-400c759e8d1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:19.379933 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f3f4d957da411e8408b67330e48d16', 'request_id': 'req-90f7189b-3046-4ee3-b145-400c759e8d1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:19.387914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f3f4d957da411e8408b67330e48d16', 'request_id': 'req-90f7189b-3046-4ee3-b145-400c759e8d1c'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:19.388692 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6861e2f3d00e443aae9b5df868602957', 'request_id': 'req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:19.397763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6861e2f3d00e443aae9b5df868602957', 'request_id': 'req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:19.403318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] The token cache is being invalidate because project 08f3f4d957da411e8408b67330e48d16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:19.403612 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:19.404450 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-90f7189b-3046-4ee3-b145-400c759e8d1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:19.405957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6861e2f3d00e443aae9b5df868602957', 'request_id': 'req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:19.406084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1390/2764] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:19 2022] DELETE /identity/v3/projects/08f3f4d957da411e8408b67330e48d16 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:19.410978 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d6d4b-0555-4851-bbc9-71c4518713fd None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.411271 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d6d4b-0555-4851-bbc9-71c4518713fd None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.411563 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d6d4b-0555-4851-bbc9-71c4518713fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.426952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] The token cache is being invalidate because project 6861e2f3d00e443aae9b5df868602957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:19.427391 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:19.428452 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-55a443db-4e41-4c5f-a0a2-f02a0f9dd028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:19.430552 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1375/2765] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:19 2022] DELETE /identity/v3/projects/6861e2f3d00e443aae9b5df868602957 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:19.433132 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-583d6d4b-0555-4851-bbc9-71c4518713fd None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:19.435449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-583d6d4b-0555-4851-bbc9-71c4518713fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:19.442717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3094c-f996-4680-876c-6db6371d01be None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.443204 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3094c-f996-4680-876c-6db6371d01be None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.443646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3094c-f996-4680-876c-6db6371d01be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.467093 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-92a3094c-f996-4680-876c-6db6371d01be None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:19.468254 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-92a3094c-f996-4680-876c-6db6371d01be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:19.502701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:19.526506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1391/2766] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:19.534790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed644f9-d2ae-44ca-b65f-51f4038f9092 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.535303 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed644f9-d2ae-44ca-b65f-51f4038f9092 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.535507 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed644f9-d2ae-44ca-b65f-51f4038f9092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.557773 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-0ed644f9-d2ae-44ca-b65f-51f4038f9092 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:19.558045 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1376/2767] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:19.559471 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-0ed644f9-d2ae-44ca-b65f-51f4038f9092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:19.567709 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1392/2768] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:19.571321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978211d-0a2e-4936-ba56-37ca634d0643 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.571321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978211d-0a2e-4936-ba56-37ca634d0643 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.571321 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978211d-0a2e-4936-ba56-37ca634d0643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.572883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed644f9-d2ae-44ca-b65f-51f4038f9092 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.582572 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed644f9-d2ae-44ca-b65f-51f4038f9092 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.589443 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44133b-1551-40d4-9421-09d07b6a4466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.590205 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44133b-1551-40d4-9421-09d07b6a4466 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.590484 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44133b-1551-40d4-9421-09d07b6a4466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.590784 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44133b-1551-40d4-9421-09d07b6a4466 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.592348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44133b-1551-40d4-9421-09d07b6a4466 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:39:19.592348 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44133b-1551-40d4-9421-09d07b6a4466 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.592752 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-c978211d-0a2e-4936-ba56-37ca634d0643 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:19.593731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44133b-1551-40d4-9421-09d07b6a4466 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.593992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c978211d-0a2e-4936-ba56-37ca634d0643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:19.606091 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1377/2769] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:19.607759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978211d-0a2e-4936-ba56-37ca634d0643 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.612890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:19.622811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1393/2770] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:39:19.623306 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978211d-0a2e-4936-ba56-37ca634d0643 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.627465 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44133b-1551-40d4-9421-09d07b6a4466 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.630071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7f2ba-92f9-4ac1-86b1-6ead617fc85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.630909 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7f2ba-92f9-4ac1-86b1-6ead617fc85b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.631246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7f2ba-92f9-4ac1-86b1-6ead617fc85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.631564 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7f2ba-92f9-4ac1-86b1-6ead617fc85b admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.632386 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7f2ba-92f9-4ac1-86b1-6ead617fc85b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:39:19.632896 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7f2ba-92f9-4ac1-86b1-6ead617fc85b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.637198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d7f2ba-92f9-4ac1-86b1-6ead617fc85b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.640985 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44133b-1551-40d4-9421-09d07b6a4466 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.647497 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76790d-1d4a-461d-af0d-d5b99fd7463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.648628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76790d-1d4a-461d-af0d-d5b99fd7463d admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.649098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76790d-1d4a-461d-af0d-d5b99fd7463d admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.649607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f76790d-1d4a-461d-af0d-d5b99fd7463d admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.651433 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76790d-1d4a-461d-af0d-d5b99fd7463d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.651763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1378/2771] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:39:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:39:19.655058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f76790d-1d4a-461d-af0d-d5b99fd7463d admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.656942 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7f2ba-92f9-4ac1-86b1-6ead617fc85b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.669775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7f2ba-92f9-4ac1-86b1-6ead617fc85b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.675856 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254aa41-2534-4346-bdc0-45893d7c753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.676540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254aa41-2534-4346-bdc0-45893d7c753f admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.676824 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254aa41-2534-4346-bdc0-45893d7c753f admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.677105 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4254aa41-2534-4346-bdc0-45893d7c753f admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.677825 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254aa41-2534-4346-bdc0-45893d7c753f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.679813 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4254aa41-2534-4346-bdc0-45893d7c753f admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.695639 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1394/2772] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:19 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:19.703875 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76790d-1d4a-461d-af0d-d5b99fd7463d admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.721282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1379/2773] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:19 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:19.722394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254aa41-2534-4346-bdc0-45893d7c753f admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.809903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4254aa41-2534-4346-bdc0-45893d7c753f admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.810235 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f76790d-1d4a-461d-af0d-d5b99fd7463d admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.816780 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccbf24-67f3-436d-b7eb-5a0f8944e753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.817717 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccbf24-67f3-436d-b7eb-5a0f8944e753 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.818099 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccbf24-67f3-436d-b7eb-5a0f8944e753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.818468 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccbf24-67f3-436d-b7eb-5a0f8944e753 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.818754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d92b9-65ef-48f8-80d2-77e1c031c796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.819421 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccbf24-67f3-436d-b7eb-5a0f8944e753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.819615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d92b9-65ef-48f8-80d2-77e1c031c796 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.820562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d92b9-65ef-48f8-80d2-77e1c031c796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.820834 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d92b9-65ef-48f8-80d2-77e1c031c796 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.821657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d92b9-65ef-48f8-80d2-77e1c031c796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.822000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccbf24-67f3-436d-b7eb-5a0f8944e753 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.823992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1d92b9-65ef-48f8-80d2-77e1c031c796 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.849472 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1380/2774] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:19 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:19.856049 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1395/2775] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:19 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:19.856269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccbf24-67f3-436d-b7eb-5a0f8944e753 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.857413 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d92b9-65ef-48f8-80d2-77e1c031c796 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.867149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccbf24-67f3-436d-b7eb-5a0f8944e753 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.872489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dddd1a-2885-448c-8252-1a5740034bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.873167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dddd1a-2885-448c-8252-1a5740034bfb admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.873473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dddd1a-2885-448c-8252-1a5740034bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.873744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dddd1a-2885-448c-8252-1a5740034bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.874489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dddd1a-2885-448c-8252-1a5740034bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.875273 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d92b9-65ef-48f8-80d2-77e1c031c796 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.876153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dddd1a-2885-448c-8252-1a5740034bfb admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.881672 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883b6ec-da76-478f-9a53-5fcdea295327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.882360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883b6ec-da76-478f-9a53-5fcdea295327 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.882685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883b6ec-da76-478f-9a53-5fcdea295327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.882974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883b6ec-da76-478f-9a53-5fcdea295327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.883727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883b6ec-da76-478f-9a53-5fcdea295327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.885687 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883b6ec-da76-478f-9a53-5fcdea295327 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.898358 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1381/2776] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:19.898794 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dddd1a-2885-448c-8252-1a5740034bfb admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.902009 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1396/2777] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:39:19.905917 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883b6ec-da76-478f-9a53-5fcdea295327 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.911415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dddd1a-2885-448c-8252-1a5740034bfb admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.917712 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883b6ec-da76-478f-9a53-5fcdea295327 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.919003 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba4f7-f40d-442e-a3e9-cfa9c3e05664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.919748 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba4f7-f40d-442e-a3e9-cfa9c3e05664 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.919977 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba4f7-f40d-442e-a3e9-cfa9c3e05664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.920313 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba4f7-f40d-442e-a3e9-cfa9c3e05664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.921034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ba4f7-f40d-442e-a3e9-cfa9c3e05664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.922709 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ba4f7-f40d-442e-a3e9-cfa9c3e05664 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.923537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9373b-fc09-46ea-9c68-0d5a7c29f0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.924253 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9373b-fc09-46ea-9c68-0d5a7c29f0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.924499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9373b-fc09-46ea-9c68-0d5a7c29f0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.926849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9373b-fc09-46ea-9c68-0d5a7c29f0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.926849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9373b-fc09-46ea-9c68-0d5a7c29f0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.927233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9373b-fc09-46ea-9c68-0d5a7c29f0c6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.937669 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1382/2778] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:19.939479 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba4f7-f40d-442e-a3e9-cfa9c3e05664 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.943067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1397/2779] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:39:19.949511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9373b-fc09-46ea-9c68-0d5a7c29f0c6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:19.951714 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba4f7-f40d-442e-a3e9-cfa9c3e05664 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.956683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d56ca-0c19-40e7-b5ef-b9847435e041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.957809 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d56ca-0c19-40e7-b5ef-b9847435e041 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.958242 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d56ca-0c19-40e7-b5ef-b9847435e041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.958528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d56ca-0c19-40e7-b5ef-b9847435e041 admin admin] PATH_INFO: `/v3/projects/ddaf35f3768446f784b287cdab59a94b/users/717f8c3cec2f4c898a8707835395fb92/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.962818 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9373b-fc09-46ea-9c68-0d5a7c29f0c6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:19.968544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea22ad-831e-48c3-a45c-dc5635ba3015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:19.969456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea22ad-831e-48c3-a45c-dc5635ba3015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:19.969742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea22ad-831e-48c3-a45c-dc5635ba3015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:19.970014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea22ad-831e-48c3-a45c-dc5635ba3015 admin admin] PATH_INFO: `/v3/projects/2d04163bef8a4f83a52200c2d171c8f5/users/0ad4fae1bf1a4c29af45a7f86a9b10ec/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:19.976608 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d56ca-0c19-40e7-b5ef-b9847435e041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddaf35f3768446f784b287cdab59a94b, user_id=717f8c3cec2f4c898a8707835395fb92, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.980148 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d56ca-0c19-40e7-b5ef-b9847435e041 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.987540 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea22ad-831e-48c3-a45c-dc5635ba3015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d04163bef8a4f83a52200c2d171c8f5, user_id=0ad4fae1bf1a4c29af45a7f86a9b10ec, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:19.990752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea22ad-831e-48c3-a45c-dc5635ba3015 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:19.994654 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1383/2780] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:19 2022] PUT /identity/v3/projects/ddaf35f3768446f784b287cdab59a94b/users/717f8c3cec2f4c898a8707835395fb92/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:20.005796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26964aea-d981-4f65-ae38-bfe0778124e9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.006066 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26964aea-d981-4f65-ae38-bfe0778124e9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.006274 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1398/2781] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:19 2022] PUT /identity/v3/projects/2d04163bef8a4f83a52200c2d171c8f5/users/0ad4fae1bf1a4c29af45a7f86a9b10ec/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:20.006420 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26964aea-d981-4f65-ae38-bfe0778124e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.019147 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80320bf-806e-487c-955c-82749847c561 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.019754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80320bf-806e-487c-955c-82749847c561 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.020358 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80320bf-806e-487c-955c-82749847c561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.039388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-26964aea-d981-4f65-ae38-bfe0778124e9 None None] MFA Rules not processed for user `717f8c3cec2f4c898a8707835395fb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:20.042133 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-26964aea-d981-4f65-ae38-bfe0778124e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:20.054461 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-f80320bf-806e-487c-955c-82749847c561 None None] MFA Rules not processed for user `0ad4fae1bf1a4c29af45a7f86a9b10ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:20.057219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-f80320bf-806e-487c-955c-82749847c561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:20.116798 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:20.118050 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:20.122007 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1384/2782] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:20.127243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26964aea-d981-4f65-ae38-bfe0778124e9 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.165601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1399/2783] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:20.165601 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80320bf-806e-487c-955c-82749847c561 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.233588 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26964aea-d981-4f65-ae38-bfe0778124e9 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.242196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b545735-9f50-4f4f-9754-92d4428aa34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.242196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b545735-9f50-4f4f-9754-92d4428aa34d admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.242196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b545735-9f50-4f4f-9754-92d4428aa34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.242196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b545735-9f50-4f4f-9754-92d4428aa34d admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.242873 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b545735-9f50-4f4f-9754-92d4428aa34d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.244585 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b545735-9f50-4f4f-9754-92d4428aa34d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.274033 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80320bf-806e-487c-955c-82749847c561 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.281748 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51ce63-94ee-49e5-8280-9a4b71572ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.283914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ce63-94ee-49e5-8280-9a4b71572ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.283914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ce63-94ee-49e5-8280-9a4b71572ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.283914 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ce63-94ee-49e5-8280-9a4b71572ab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.287998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51ce63-94ee-49e5-8280-9a4b71572ab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.287998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51ce63-94ee-49e5-8280-9a4b71572ab6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.298771 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1385/2784] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:20 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:20.299910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b545735-9f50-4f4f-9754-92d4428aa34d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.336967 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1400/2785] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:20 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:20.341374 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51ce63-94ee-49e5-8280-9a4b71572ab6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.416462 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b545735-9f50-4f4f-9754-92d4428aa34d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.433546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51ce63-94ee-49e5-8280-9a4b71572ab6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.439678 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91d717-d208-46e7-ae9a-683c6a7e303f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.440657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91d717-d208-46e7-ae9a-683c6a7e303f admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.440947 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91d717-d208-46e7-ae9a-683c6a7e303f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.441230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91d717-d208-46e7-ae9a-683c6a7e303f admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.441938 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91d717-d208-46e7-ae9a-683c6a7e303f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.443939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91d717-d208-46e7-ae9a-683c6a7e303f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.470996 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1401/2786] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:20 2022] POST /identity/v3/users => generated 298 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:20.476223 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91d717-d208-46e7-ae9a-683c6a7e303f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.492080 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d793c14-a290-4793-b688-da1153ad915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.492904 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793c14-a290-4793-b688-da1153ad915c admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.493227 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793c14-a290-4793-b688-da1153ad915c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.493550 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d793c14-a290-4793-b688-da1153ad915c admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.494271 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d793c14-a290-4793-b688-da1153ad915c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.495798 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91d717-d208-46e7-ae9a-683c6a7e303f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.496317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d793c14-a290-4793-b688-da1153ad915c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.502473 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03c60e-b822-4181-ba76-5a02b5563ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.503514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03c60e-b822-4181-ba76-5a02b5563ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.503834 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03c60e-b822-4181-ba76-5a02b5563ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.504121 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f03c60e-b822-4181-ba76-5a02b5563ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.506118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03c60e-b822-4181-ba76-5a02b5563ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.508587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f03c60e-b822-4181-ba76-5a02b5563ba4 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.521337 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1386/2787] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:20 2022] POST /identity/v3/users => generated 290 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:20.521832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1402/2788] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:20.525854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d793c14-a290-4793-b688-da1153ad915c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.526446 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03c60e-b822-4181-ba76-5a02b5563ba4 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.538194 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d793c14-a290-4793-b688-da1153ad915c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.539294 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f03c60e-b822-4181-ba76-5a02b5563ba4 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.544423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b45da-e1ae-4533-8519-db7fdd31e67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.545325 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b45da-e1ae-4533-8519-db7fdd31e67e admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.545899 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b45da-e1ae-4533-8519-db7fdd31e67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.546028 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b45da-e1ae-4533-8519-db7fdd31e67e admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.546888 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b45da-e1ae-4533-8519-db7fdd31e67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.548492 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46531bf8-78c0-4a8a-bc18-b123bb794e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.548742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743b45da-e1ae-4533-8519-db7fdd31e67e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.550146 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531bf8-78c0-4a8a-bc18-b123bb794e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.550685 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531bf8-78c0-4a8a-bc18-b123bb794e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.551187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531bf8-78c0-4a8a-bc18-b123bb794e6f admin admin] PATH_INFO: `/v3/projects/80772998ddcd4abc8ae36b69e99a08df/users/85b9daa53a404771a5bdbf40ea8c3315/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.561213 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1387/2789] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:39:20.566026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b45da-e1ae-4533-8519-db7fdd31e67e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.572790 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46531bf8-78c0-4a8a-bc18-b123bb794e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=80772998ddcd4abc8ae36b69e99a08df, user_id=85b9daa53a404771a5bdbf40ea8c3315, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.576164 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46531bf8-78c0-4a8a-bc18-b123bb794e6f admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.581245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b45da-e1ae-4533-8519-db7fdd31e67e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.587376 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebfda4-4bf1-4230-b7f8-f9a806584a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.588331 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebfda4-4bf1-4230-b7f8-f9a806584a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.589115 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebfda4-4bf1-4230-b7f8-f9a806584a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.589418 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebfda4-4bf1-4230-b7f8-f9a806584a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.590173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebfda4-4bf1-4230-b7f8-f9a806584a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.591850 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebfda4-4bf1-4230-b7f8-f9a806584a98 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.594199 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1403/2790] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:20 2022] PUT /identity/v3/projects/80772998ddcd4abc8ae36b69e99a08df/users/85b9daa53a404771a5bdbf40ea8c3315/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:20.598269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46531bf8-78c0-4a8a-bc18-b123bb794e6f admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.599454 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:20.608596 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1388/2791] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:20.612857 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebfda4-4bf1-4230-b7f8-f9a806584a98 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.625578 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebfda4-4bf1-4230-b7f8-f9a806584a98 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.630927 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:20.684604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46531bf8-78c0-4a8a-bc18-b123bb794e6f admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.690382 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fcfbf-fcbc-458a-be34-f805dcfc6fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.691187 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fcfbf-fcbc-458a-be34-f805dcfc6fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.691548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fcfbf-fcbc-458a-be34-f805dcfc6fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.691883 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fcfbf-fcbc-458a-be34-f805dcfc6fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.692585 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fcfbf-fcbc-458a-be34-f805dcfc6fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.694230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1fcfbf-fcbc-458a-be34-f805dcfc6fd5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.694730 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cc954-e381-4f35-ad66-3d347a61a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.695656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cc954-e381-4f35-ad66-3d347a61a77d admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.695939 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cc954-e381-4f35-ad66-3d347a61a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.696217 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cc954-e381-4f35-ad66-3d347a61a77d admin admin] PATH_INFO: `/v3/projects/c899a82b20fa4aa0b3ad08e65a28face/users/c181982acedc4a16b1dba798a9d19e18/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.705881 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1404/2792] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:39:20.711062 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fcfbf-fcbc-458a-be34-f805dcfc6fd5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.714960 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6cc954-e381-4f35-ad66-3d347a61a77d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c899a82b20fa4aa0b3ad08e65a28face, user_id=c181982acedc4a16b1dba798a9d19e18, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.717679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6cc954-e381-4f35-ad66-3d347a61a77d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.723242 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fcfbf-fcbc-458a-be34-f805dcfc6fd5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.730086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d2f87-3409-4ca4-b408-65196e8b5c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.731023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d2f87-3409-4ca4-b408-65196e8b5c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.731344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d2f87-3409-4ca4-b408-65196e8b5c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.731576 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d2f87-3409-4ca4-b408-65196e8b5c27 admin admin] PATH_INFO: `/v3/projects/80772998ddcd4abc8ae36b69e99a08df/users/85b9daa53a404771a5bdbf40ea8c3315/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.732975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1389/2793] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:20 2022] PUT /identity/v3/projects/c899a82b20fa4aa0b3ad08e65a28face/users/c181982acedc4a16b1dba798a9d19e18/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:20.734017 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d2f87-3409-4ca4-b408-65196e8b5c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80772998ddcd4abc8ae36b69e99a08df, user_id=85b9daa53a404771a5bdbf40ea8c3315, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.736661 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d2f87-3409-4ca4-b408-65196e8b5c27 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.750322 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e442a5-6fee-46cb-a928-4f67d30bd92c None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.750741 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e442a5-6fee-46cb-a928-4f67d30bd92c None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.751063 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e442a5-6fee-46cb-a928-4f67d30bd92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.754827 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1405/2794] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:20 2022] PUT /identity/v3/projects/80772998ddcd4abc8ae36b69e99a08df/users/85b9daa53a404771a5bdbf40ea8c3315/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:20.768925 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13e8ba-5957-42f7-9f53-f58f2d7aa8bc None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.769951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13e8ba-5957-42f7-9f53-f58f2d7aa8bc None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.770440 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13e8ba-5957-42f7-9f53-f58f2d7aa8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.785993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-f3e442a5-6fee-46cb-a928-4f67d30bd92c None None] MFA Rules not processed for user `c181982acedc4a16b1dba798a9d19e18`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:20.787285 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f3e442a5-6fee-46cb-a928-4f67d30bd92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:20.814615 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-eb13e8ba-5957-42f7-9f53-f58f2d7aa8bc None None] MFA Rules not processed for user `85b9daa53a404771a5bdbf40ea8c3315`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:20.815972 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eb13e8ba-5957-42f7-9f53-f58f2d7aa8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:20.869471 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1390/2795] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:20.873938 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e442a5-6fee-46cb-a928-4f67d30bd92c None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.900198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1406/2796] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:20 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:20.911704 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13e8ba-5957-42f7-9f53-f58f2d7aa8bc None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:20.913352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eb13e8ba-5957-42f7-9f53-f58f2d7aa8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:20.946422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e442a5-6fee-46cb-a928-4f67d30bd92c None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.951645 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda197ba-5a28-4ca7-8202-2407308d9af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:20.952495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda197ba-5a28-4ca7-8202-2407308d9af6 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:20.952947 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda197ba-5a28-4ca7-8202-2407308d9af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:20.953245 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda197ba-5a28-4ca7-8202-2407308d9af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:20.954136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda197ba-5a28-4ca7-8202-2407308d9af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:20.956122 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda197ba-5a28-4ca7-8202-2407308d9af6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:20.977688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:20.984782 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13e8ba-5957-42f7-9f53-f58f2d7aa8bc None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:20.989805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1391/2797] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:20 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:20.990686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda197ba-5a28-4ca7-8202-2407308d9af6 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:21.039975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:21.047331 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7939e8-9d95-41cd-b91e-1d96faeefa7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:21.048165 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7939e8-9d95-41cd-b91e-1d96faeefa7f service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:21.048776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7939e8-9d95-41cd-b91e-1d96faeefa7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:21.049067 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7939e8-9d95-41cd-b91e-1d96faeefa7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:21.059577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda197ba-5a28-4ca7-8202-2407308d9af6 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:21.064243 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258ab33-4fe2-4607-906a-6bec0075d7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:21.064910 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258ab33-4fe2-4607-906a-6bec0075d7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:21.065196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258ab33-4fe2-4607-906a-6bec0075d7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:21.065475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258ab33-4fe2-4607-906a-6bec0075d7a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:21.066166 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258ab33-4fe2-4607-906a-6bec0075d7a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:21.068126 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258ab33-4fe2-4607-906a-6bec0075d7a2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:21.089110 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1392/2798] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:20 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:21.093049 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258ab33-4fe2-4607-906a-6bec0075d7a2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:21.102507 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258ab33-4fe2-4607-906a-6bec0075d7a2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:21.107626 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47651855-55f2-4c8d-b97b-78a8d5d19f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:21.108384 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47651855-55f2-4c8d-b97b-78a8d5d19f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:21.108563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47651855-55f2-4c8d-b97b-78a8d5d19f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:21.108842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47651855-55f2-4c8d-b97b-78a8d5d19f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:21.109557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47651855-55f2-4c8d-b97b-78a8d5d19f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:21.111179 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47651855-55f2-4c8d-b97b-78a8d5d19f2e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:21.111783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7939e8-9d95-41cd-b91e-1d96faeefa7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:21.113282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7939e8-9d95-41cd-b91e-1d96faeefa7f service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:21.121760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1393/2799] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:39:21.124079 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1407/2800] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:21.125913 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47651855-55f2-4c8d-b97b-78a8d5d19f2e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:21.135796 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47651855-55f2-4c8d-b97b-78a8d5d19f2e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:21.140806 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c43ae5-d399-412b-924c-693bfe97a119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:21.141682 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c43ae5-d399-412b-924c-693bfe97a119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:21.141968 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c43ae5-d399-412b-924c-693bfe97a119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:21.142372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c43ae5-d399-412b-924c-693bfe97a119 admin admin] PATH_INFO: `/v3/projects/1f7abd07d7f0430fbce9da2c9d5c75e4/users/102e475b2ca44358a8b38a4c87f7b794/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:21.157842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c43ae5-d399-412b-924c-693bfe97a119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7abd07d7f0430fbce9da2c9d5c75e4, user_id=102e475b2ca44358a8b38a4c87f7b794, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:21.160542 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c43ae5-d399-412b-924c-693bfe97a119 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:21.174903 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1394/2801] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:21 2022] PUT /identity/v3/projects/1f7abd07d7f0430fbce9da2c9d5c75e4/users/102e475b2ca44358a8b38a4c87f7b794/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:21.178880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7939e8-9d95-41cd-b91e-1d96faeefa7f service cinder] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:21.190898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7939e8-9d95-41cd-b91e-1d96faeefa7f service cinder] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:21.250841 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3ec4e-a3a7-4727-905e-6616aac5037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:21.251432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3ec4e-a3a7-4727-905e-6616aac5037a admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:21.251722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3ec4e-a3a7-4727-905e-6616aac5037a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:21.252003 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3ec4e-a3a7-4727-905e-6616aac5037a admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:21.252741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3ec4e-a3a7-4727-905e-6616aac5037a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:21.254384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3ec4e-a3a7-4727-905e-6616aac5037a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:21.268915 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1408/2802] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:21.270099 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c43ae5-d399-412b-924c-693bfe97a119 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:21.281547 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c43ae5-d399-412b-924c-693bfe97a119 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:21.286237 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f4d5a-dd01-478f-97f9-12f286d90ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:21.287168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f4d5a-dd01-478f-97f9-12f286d90ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:21.287459 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f4d5a-dd01-478f-97f9-12f286d90ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:21.287740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f4d5a-dd01-478f-97f9-12f286d90ad5 admin admin] PATH_INFO: `/v3/projects/1f7abd07d7f0430fbce9da2c9d5c75e4/users/102e475b2ca44358a8b38a4c87f7b794/roles/a48bc1eaca7e4ade827ca5b68110af29` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:21.289777 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f4d5a-dd01-478f-97f9-12f286d90ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7abd07d7f0430fbce9da2c9d5c75e4, user_id=102e475b2ca44358a8b38a4c87f7b794, role_id=a48bc1eaca7e4ade827ca5b68110af29)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:21.293240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f4d5a-dd01-478f-97f9-12f286d90ad5 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:21.304034 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1395/2803] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:21 2022] PUT /identity/v3/projects/1f7abd07d7f0430fbce9da2c9d5c75e4/users/102e475b2ca44358a8b38a4c87f7b794/roles/a48bc1eaca7e4ade827ca5b68110af29 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:21.315546 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21e6b1-80ab-456b-8494-816cd594f215 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:21.315861 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21e6b1-80ab-456b-8494-816cd594f215 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:21.316126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21e6b1-80ab-456b-8494-816cd594f215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:21.348162 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-bd21e6b1-80ab-456b-8494-816cd594f215 None None] MFA Rules not processed for user `102e475b2ca44358a8b38a4c87f7b794`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:21.349700 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bd21e6b1-80ab-456b-8494-816cd594f215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:21.449652 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1409/2804] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:21.463277 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f4d5a-dd01-478f-97f9-12f286d90ad5 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:21.465068 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c50f4d5a-dd01-478f-97f9-12f286d90ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:21.559713 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f4d5a-dd01-478f-97f9-12f286d90ad5 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:21.563594 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f09238-3d66-4182-af44-a2c1bfa40aff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:21.564289 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f09238-3d66-4182-af44-a2c1bfa40aff service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:21.564553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f09238-3d66-4182-af44-a2c1bfa40aff service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:21.564821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f09238-3d66-4182-af44-a2c1bfa40aff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:21.575021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f09238-3d66-4182-af44-a2c1bfa40aff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:21.576523 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f09238-3d66-4182-af44-a2c1bfa40aff service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:21.588050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1396/2805] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:21.632657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21e6b1-80ab-456b-8494-816cd594f215 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:21.643033 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21e6b1-80ab-456b-8494-816cd594f215 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:21.647148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6c6a8-67df-47ae-9d9d-6403b01f5525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:21.647812 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6c6a8-67df-47ae-9d9d-6403b01f5525 service cinder] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:21.648260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6c6a8-67df-47ae-9d9d-6403b01f5525 service cinder] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:21.648643 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6c6a8-67df-47ae-9d9d-6403b01f5525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:21.671690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:21.716845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6c6a8-67df-47ae-9d9d-6403b01f5525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:21.718312 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6c6a8-67df-47ae-9d9d-6403b01f5525 service cinder] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:21.727921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1410/2806] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:24.725548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f09238-3d66-4182-af44-a2c1bfa40aff service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:24.779792 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:24.800030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f09238-3d66-4182-af44-a2c1bfa40aff service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:24.805368 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:24.806263 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:24.806554 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:24.807736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] PATH_INFO: `/v3/users/717f8c3cec2f4c898a8707835395fb92` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:24.809219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=717f8c3cec2f4c898a8707835395fb92)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:24.811232 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:24.865570 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '717f8c3cec2f4c898a8707835395fb92'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:24.877116 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '717f8c3cec2f4c898a8707835395fb92'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:24.886171 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '717f8c3cec2f4c898a8707835395fb92'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:24.899290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '717f8c3cec2f4c898a8707835395fb92'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:24.900026 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '717f8c3cec2f4c898a8707835395fb92'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:24.906493 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1397/2807] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:24 2022] DELETE /identity/v3/users/717f8c3cec2f4c898a8707835395fb92 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:24.913514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e23f5-8a2f-4be6-8d9b-b72c4e32231f None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:24.914075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e23f5-8a2f-4be6-8d9b-b72c4e32231f None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:24.914075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e23f5-8a2f-4be6-8d9b-b72c4e32231f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:24.933295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a53e23f5-8a2f-4be6-8d9b-b72c4e32231f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:24.934532 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a53e23f5-8a2f-4be6-8d9b-b72c4e32231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:24.992711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1411/2808] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:25.003383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:25.004826 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:25.067103 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0368d-9f87-41dc-b32c-92f02b43e716 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:25.070761 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f11ce-e82b-498b-951c-5abffa77810c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:25.071417 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f11ce-e82b-498b-951c-5abffa77810c service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:25.071705 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f11ce-e82b-498b-951c-5abffa77810c service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:25.071987 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21f11ce-e82b-498b-951c-5abffa77810c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:25.081188 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f11ce-e82b-498b-951c-5abffa77810c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:25.082663 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21f11ce-e82b-498b-951c-5abffa77810c service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:25.093223 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1398/2809] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:25.394339 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e23f5-8a2f-4be6-8d9b-b72c4e32231f None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:25.397998 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a53e23f5-8a2f-4be6-8d9b-b72c4e32231f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:25.406759 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f11ce-e82b-498b-951c-5abffa77810c service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:25.408577 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c21f11ce-e82b-498b-951c-5abffa77810c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:25.426870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e23f5-8a2f-4be6-8d9b-b72c4e32231f None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:25.434722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:25.435360 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:25.435739 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:25.436154 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] PATH_INFO: `/v3/projects/ddaf35f3768446f784b287cdab59a94b` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:25.437763 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddaf35f3768446f784b287cdab59a94b)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:25.440116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:25.489519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21f11ce-e82b-498b-951c-5abffa77810c service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:25.495656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddaf35f3768446f784b287cdab59a94b', 'request_id': 'req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:25.503179 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddaf35f3768446f784b287cdab59a94b', 'request_id': 'req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:25.506197 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:25.510775 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddaf35f3768446f784b287cdab59a94b', 'request_id': 'req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:25.522269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] The token cache is being invalidate because project ddaf35f3768446f784b287cdab59a94b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:25.522564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:25.523365 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:25.525114 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1412/2810] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:25 2022] DELETE /identity/v3/projects/ddaf35f3768446f784b287cdab59a94b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:25.529557 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:25.530961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:25.545688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffdd7b-9d92-4bca-8423-dabf14155053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:25.546380 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffdd7b-9d92-4bca-8423-dabf14155053 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:25.546693 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffdd7b-9d92-4bca-8423-dabf14155053 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:25.546975 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffdd7b-9d92-4bca-8423-dabf14155053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:25.548787 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d8ffdd7b-9d92-4bca-8423-dabf14155053 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:25.579682 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:25.603021 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130435db-0ce3-4f00-8ff7-d9056e9cfaa5 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:25.609117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:25.610604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:25.610919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:25.611204 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] PATH_INFO: `/v3/users/c181982acedc4a16b1dba798a9d19e18` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:25.612463 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c181982acedc4a16b1dba798a9d19e18)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:25.614449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:25.620965 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffdd7b-9d92-4bca-8423-dabf14155053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:25.622710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffdd7b-9d92-4bca-8423-dabf14155053 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:25.633274 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1399/2811] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:25.673491 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c181982acedc4a16b1dba798a9d19e18'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:25.689334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c181982acedc4a16b1dba798a9d19e18'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:25.690082 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c181982acedc4a16b1dba798a9d19e18'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:25.700581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c181982acedc4a16b1dba798a9d19e18'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:25.710273 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c181982acedc4a16b1dba798a9d19e18'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:25.717723 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1413/2812] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:25 2022] DELETE /identity/v3/users/c181982acedc4a16b1dba798a9d19e18 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:26.060726 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffdd7b-9d92-4bca-8423-dabf14155053 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:26.071967 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffdd7b-9d92-4bca-8423-dabf14155053 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:26.145572 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:26.146481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:26.146804 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:26.147092 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] PATH_INFO: `/v3/projects/c899a82b20fa4aa0b3ad08e65a28face` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:26.148957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c899a82b20fa4aa0b3ad08e65a28face)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:26.151656 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:26.201555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c899a82b20fa4aa0b3ad08e65a28face', 'request_id': 'req-39df6a45-150c-4318-a385-9b4b7d847eb2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.207025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c899a82b20fa4aa0b3ad08e65a28face', 'request_id': 'req-39df6a45-150c-4318-a385-9b4b7d847eb2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.213498 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c899a82b20fa4aa0b3ad08e65a28face', 'request_id': 'req-39df6a45-150c-4318-a385-9b4b7d847eb2'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.226267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] The token cache is being invalidate because project c899a82b20fa4aa0b3ad08e65a28face was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:26.226805 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:26.227270 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.228039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.230186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1400/2813] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:26 2022] DELETE /identity/v3/projects/c899a82b20fa4aa0b3ad08e65a28face => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:26.234421 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:26.236014 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:26.250171 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c329d-3eb1-4b01-b64f-8d1654972749 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:26.307870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066be09-a42f-49a6-8d25-727772d38ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:26.308767 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066be09-a42f-49a6-8d25-727772d38ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:26.309061 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066be09-a42f-49a6-8d25-727772d38ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:26.309341 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066be09-a42f-49a6-8d25-727772d38ba6 admin admin] PATH_INFO: `/v3/users/102e475b2ca44358a8b38a4c87f7b794` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:26.310564 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066be09-a42f-49a6-8d25-727772d38ba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=102e475b2ca44358a8b38a4c87f7b794)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:26.312544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066be09-a42f-49a6-8d25-727772d38ba6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:26.357930 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2066be09-a42f-49a6-8d25-727772d38ba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '102e475b2ca44358a8b38a4c87f7b794'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.370684 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2066be09-a42f-49a6-8d25-727772d38ba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '102e475b2ca44358a8b38a4c87f7b794'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.371400 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2066be09-a42f-49a6-8d25-727772d38ba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '102e475b2ca44358a8b38a4c87f7b794'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.373084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:26.379737 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2066be09-a42f-49a6-8d25-727772d38ba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '102e475b2ca44358a8b38a4c87f7b794'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.389002 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-2066be09-a42f-49a6-8d25-727772d38ba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '102e475b2ca44358a8b38a4c87f7b794'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.396222 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1414/2814] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:26 2022] DELETE /identity/v3/users/102e475b2ca44358a8b38a4c87f7b794 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:26.493747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:26.495283 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:26.509316 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df6a45-150c-4318-a385-9b4b7d847eb2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:26.576842 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:26.577715 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:26.577997 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:26.578272 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] PATH_INFO: `/v3/users/8728f2d0b848424d86b24c794c8d3388` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:26.579595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8728f2d0b848424d86b24c794c8d3388)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:26.581557 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:26.636030 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8728f2d0b848424d86b24c794c8d3388'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.644399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8728f2d0b848424d86b24c794c8d3388'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.656058 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8728f2d0b848424d86b24c794c8d3388'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.676326 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8728f2d0b848424d86b24c794c8d3388'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.677051 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8728f2d0b848424d86b24c794c8d3388'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.684423 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1401/2815] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:26 2022] DELETE /identity/v3/users/8728f2d0b848424d86b24c794c8d3388 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:26.689359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc86f20-7cac-4230-adbe-7930a7608eb2 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:26.689734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc86f20-7cac-4230-adbe-7930a7608eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:26.690136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc86f20-7cac-4230-adbe-7930a7608eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:26.707777 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:26.709235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:26.718749 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-3fc86f20-7cac-4230-adbe-7930a7608eb2 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:26.720544 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3fc86f20-7cac-4230-adbe-7930a7608eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:26.791388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03291bf-d98d-44f1-bbf0-81979cfc1a7c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:26.797376 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:26.798261 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:26.798653 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:26.798843 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] PATH_INFO: `/v3/projects/1f7abd07d7f0430fbce9da2c9d5c75e4` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:26.800070 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f7abd07d7f0430fbce9da2c9d5c75e4)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:26.802107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:26.803746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1415/2816] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:26.815804 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc86f20-7cac-4230-adbe-7930a7608eb2 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:26.818054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-3fc86f20-7cac-4230-adbe-7930a7608eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:26.837127 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:26.863327 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f7abd07d7f0430fbce9da2c9d5c75e4', 'request_id': 'req-a93d43fd-934c-4b16-8999-0ab4161843df'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.869075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f7abd07d7f0430fbce9da2c9d5c75e4', 'request_id': 'req-a93d43fd-934c-4b16-8999-0ab4161843df'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.876400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f7abd07d7f0430fbce9da2c9d5c75e4', 'request_id': 'req-a93d43fd-934c-4b16-8999-0ab4161843df'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.881375 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc86f20-7cac-4230-adbe-7930a7608eb2 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:26.889527 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] The token cache is being invalidate because project 1f7abd07d7f0430fbce9da2c9d5c75e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:26.890268 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.890952 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a93d43fd-934c-4b16-8999-0ab4161843df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:26.892736 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1402/2817] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:26 2022] DELETE /identity/v3/projects/1f7abd07d7f0430fbce9da2c9d5c75e4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:26.898623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e413cb0-cf0d-4a66-9171-e92eccfc602f None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:26.898999 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e413cb0-cf0d-4a66-9171-e92eccfc602f None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:26.899348 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e413cb0-cf0d-4a66-9171-e92eccfc602f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:26.918814 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-6e413cb0-cf0d-4a66-9171-e92eccfc602f None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:26.920290 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6e413cb0-cf0d-4a66-9171-e92eccfc602f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:26.937975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb245a52-e957-4e18-bbc1-c12dd41f9e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:26.938673 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb245a52-e957-4e18-bbc1-c12dd41f9e65 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:26.938961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb245a52-e957-4e18-bbc1-c12dd41f9e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:26.939240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb245a52-e957-4e18-bbc1-c12dd41f9e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:26.941585 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-fb245a52-e957-4e18-bbc1-c12dd41f9e65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:26.998133 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:26.998740 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1403/2818] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:27.016958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d1d3e-0b51-4b40-a09f-7c041a6320f7 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.022495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d1d3e-0b51-4b40-a09f-7c041a6320f7 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.022495 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d1d3e-0b51-4b40-a09f-7c041a6320f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.024706 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb245a52-e957-4e18-bbc1-c12dd41f9e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.028536 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb245a52-e957-4e18-bbc1-c12dd41f9e65 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.039979 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1416/2819] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:39:27.050402 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b12d1d3e-0b51-4b40-a09f-7c041a6320f7 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:27.052946 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b12d1d3e-0b51-4b40-a09f-7c041a6320f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:27.062310 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1404/2820] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:27.066316 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb245a52-e957-4e18-bbc1-c12dd41f9e65 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.078799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb245a52-e957-4e18-bbc1-c12dd41f9e65 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.084149 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168a53f-7d8d-4b75-86a2-166da27586e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.085120 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a53f-7d8d-4b75-86a2-166da27586e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.085505 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a53f-7d8d-4b75-86a2-166da27586e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.085890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168a53f-7d8d-4b75-86a2-166da27586e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.086786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168a53f-7d8d-4b75-86a2-166da27586e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:39:27.087807 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168a53f-7d8d-4b75-86a2-166da27586e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.089982 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e168a53f-7d8d-4b75-86a2-166da27586e9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.107783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1417/2821] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:39:27.112681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d1d3e-0b51-4b40-a09f-7c041a6320f7 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.137764 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d1d3e-0b51-4b40-a09f-7c041a6320f7 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.148142 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca30c7-9341-4049-a6aa-3bbd8f39846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.149233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca30c7-9341-4049-a6aa-3bbd8f39846b admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.150057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca30c7-9341-4049-a6aa-3bbd8f39846b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.150057 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca30c7-9341-4049-a6aa-3bbd8f39846b admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.151514 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca30c7-9341-4049-a6aa-3bbd8f39846b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.153840 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca30c7-9341-4049-a6aa-3bbd8f39846b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.199184 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1405/2822] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:27 2022] POST /identity/v3/projects => generated 369 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:27.204417 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168a53f-7d8d-4b75-86a2-166da27586e9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.217468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168a53f-7d8d-4b75-86a2-166da27586e9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.297054 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5fada-1e4d-4da4-89d1-0285ccf2ebfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.297882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5fada-1e4d-4da4-89d1-0285ccf2ebfd admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.298248 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5fada-1e4d-4da4-89d1-0285ccf2ebfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.299355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc5fada-1e4d-4da4-89d1-0285ccf2ebfd admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.300441 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5fada-1e4d-4da4-89d1-0285ccf2ebfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.302823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc5fada-1e4d-4da4-89d1-0285ccf2ebfd admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.334301 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1418/2823] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:27 2022] POST /identity/v3/users => generated 285 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:27.339094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca30c7-9341-4049-a6aa-3bbd8f39846b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.353872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca30c7-9341-4049-a6aa-3bbd8f39846b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.360553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7d217-25cc-453f-bb21-57d2ef37b8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.361136 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7d217-25cc-453f-bb21-57d2ef37b8c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.361733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7d217-25cc-453f-bb21-57d2ef37b8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.361733 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7d217-25cc-453f-bb21-57d2ef37b8c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.362673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7d217-25cc-453f-bb21-57d2ef37b8c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.364316 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7d217-25cc-453f-bb21-57d2ef37b8c2 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.386118 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1406/2824] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:27.388696 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5fada-1e4d-4da4-89d1-0285ccf2ebfd admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.407344 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc5fada-1e4d-4da4-89d1-0285ccf2ebfd admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.412769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb475ec9-337a-4bfd-b8bb-e97d2ba926c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.413560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb475ec9-337a-4bfd-b8bb-e97d2ba926c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.413909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb475ec9-337a-4bfd-b8bb-e97d2ba926c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.414254 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb475ec9-337a-4bfd-b8bb-e97d2ba926c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.415225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb475ec9-337a-4bfd-b8bb-e97d2ba926c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.416961 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb475ec9-337a-4bfd-b8bb-e97d2ba926c6 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.430552 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1419/2825] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:39:27.435773 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7d217-25cc-453f-bb21-57d2ef37b8c2 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.446408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7d217-25cc-453f-bb21-57d2ef37b8c2 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.452706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7953e-1210-44ed-b58c-3cdec3b4093e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.453710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7953e-1210-44ed-b58c-3cdec3b4093e admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.454001 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7953e-1210-44ed-b58c-3cdec3b4093e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.454324 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7953e-1210-44ed-b58c-3cdec3b4093e admin admin] PATH_INFO: `/v3/projects/45afcd5f1d304b6e8f8b3b44d61f2b97/users/7a87e9c648e34e7c970f5cee8b8a5e06/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.464495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb475ec9-337a-4bfd-b8bb-e97d2ba926c6 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.466506 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-eb475ec9-337a-4bfd-b8bb-e97d2ba926c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:27.481620 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7953e-1210-44ed-b58c-3cdec3b4093e admin admin] RBAC: Authorizing `identity:create_grant(project_id=45afcd5f1d304b6e8f8b3b44d61f2b97, user_id=7a87e9c648e34e7c970f5cee8b8a5e06, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.483386 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb475ec9-337a-4bfd-b8bb-e97d2ba926c6 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.484706 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7953e-1210-44ed-b58c-3cdec3b4093e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.488939 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.491174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.491494 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.491829 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] PATH_INFO: `/v3/projects/02de469128b44ac4924b1f5aa7cb6ddb` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.493896 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02de469128b44ac4924b1f5aa7cb6ddb)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.496810 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.501233 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1407/2826] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:27 2022] PUT /identity/v3/projects/45afcd5f1d304b6e8f8b3b44d61f2b97/users/7a87e9c648e34e7c970f5cee8b8a5e06/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:27.554420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02de469128b44ac4924b1f5aa7cb6ddb', 'request_id': 'req-31d7bc23-fe26-479c-b5f9-1e1cc3984521'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:27.563084 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02de469128b44ac4924b1f5aa7cb6ddb', 'request_id': 'req-31d7bc23-fe26-479c-b5f9-1e1cc3984521'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:27.570550 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0657ee7-abe5-40ad-8e54-6ffabef56ee9 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.571119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0657ee7-abe5-40ad-8e54-6ffabef56ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.571288 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02de469128b44ac4924b1f5aa7cb6ddb', 'request_id': 'req-31d7bc23-fe26-479c-b5f9-1e1cc3984521'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:27.571411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0657ee7-abe5-40ad-8e54-6ffabef56ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.582349 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] The token cache is being invalidate because project 02de469128b44ac4924b1f5aa7cb6ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:27.582638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:27.583259 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-31d7bc23-fe26-479c-b5f9-1e1cc3984521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:27.586628 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1420/2827] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:27 2022] DELETE /identity/v3/projects/02de469128b44ac4924b1f5aa7cb6ddb => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:27.591575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f6e7b-d10d-41ad-894b-d94274f59b30 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.591863 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f6e7b-d10d-41ad-894b-d94274f59b30 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.592230 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3f6e7b-d10d-41ad-894b-d94274f59b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.602524 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-e0657ee7-abe5-40ad-8e54-6ffabef56ee9 None None] MFA Rules not processed for user `7a87e9c648e34e7c970f5cee8b8a5e06`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:27.603837 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-e0657ee7-abe5-40ad-8e54-6ffabef56ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:27.610953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-1d3f6e7b-d10d-41ad-894b-d94274f59b30 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:27.612189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1d3f6e7b-d10d-41ad-894b-d94274f59b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:27.641537 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:27.647405 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:27.678427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1408/2828] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:27.684921 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1421/2829] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:27.694161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca40cf6-a740-4423-b622-326294b27ddd None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.694374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca40cf6-a740-4423-b622-326294b27ddd None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.694747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca40cf6-a740-4423-b622-326294b27ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.699561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f6e7b-d10d-41ad-894b-d94274f59b30 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.700950 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-1d3f6e7b-d10d-41ad-894b-d94274f59b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:27.716540 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-3ca40cf6-a740-4423-b622-326294b27ddd None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:27.717745 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-3ca40cf6-a740-4423-b622-326294b27ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:27.729278 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1409/2830] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:27.733485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca40cf6-a740-4423-b622-326294b27ddd None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.745829 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca40cf6-a740-4423-b622-326294b27ddd None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.751941 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f06f31-d082-4816-a34f-8a7a8b72053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.752765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f06f31-d082-4816-a34f-8a7a8b72053d admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.753135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f06f31-d082-4816-a34f-8a7a8b72053d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.753623 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f06f31-d082-4816-a34f-8a7a8b72053d admin admin] PATH_INFO: `/v3/domains` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.754894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f06f31-d082-4816-a34f-8a7a8b72053d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80225) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:39:27.755210 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f06f31-d082-4816-a34f-8a7a8b72053d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.756971 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f06f31-d082-4816-a34f-8a7a8b72053d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.766560 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3f6e7b-d10d-41ad-894b-d94274f59b30 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.770747 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1410/2831] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 08:39:27.773948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73e126-e657-4328-8c08-377c7df622b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3778997232d44cd3a3c4fb3d53e4db53', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.775216 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f06f31-d082-4816-a34f-8a7a8b72053d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.775427 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e126-e657-4328-8c08-377c7df622b1 service glance] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.776451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e126-e657-4328-8c08-377c7df622b1 service glance] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.777229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee73e126-e657-4328-8c08-377c7df622b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.790073 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f06f31-d082-4816-a34f-8a7a8b72053d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.792974 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73e126-e657-4328-8c08-377c7df622b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.795388 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee73e126-e657-4328-8c08-377c7df622b1 service glance] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.796819 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a746f2b-69f6-4d1e-b9b3-0d7eaf5cf517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.797553 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746f2b-69f6-4d1e-b9b3-0d7eaf5cf517 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.797845 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746f2b-69f6-4d1e-b9b3-0d7eaf5cf517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.798129 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a746f2b-69f6-4d1e-b9b3-0d7eaf5cf517 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.799048 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a746f2b-69f6-4d1e-b9b3-0d7eaf5cf517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.801169 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a746f2b-69f6-4d1e-b9b3-0d7eaf5cf517 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.806247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1422/2832] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 08:39:27.830326 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1411/2833] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:27 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:27.834562 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73e126-e657-4328-8c08-377c7df622b1 service glance] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.849813 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee73e126-e657-4328-8c08-377c7df622b1 service glance] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.914467 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b430d-e47a-4410-ab04-cefd8433790e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.915324 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b430d-e47a-4410-ab04-cefd8433790e admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.915613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b430d-e47a-4410-ab04-cefd8433790e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.915893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b430d-e47a-4410-ab04-cefd8433790e admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.916797 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b430d-e47a-4410-ab04-cefd8433790e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.919211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b430d-e47a-4410-ab04-cefd8433790e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.944075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1423/2834] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:27 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:27.945466 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a746f2b-69f6-4d1e-b9b3-0d7eaf5cf517 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:27.956568 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a746f2b-69f6-4d1e-b9b3-0d7eaf5cf517 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:27.961317 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91f2f6-8430-4735-93b1-b35efd08ed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:27.961981 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91f2f6-8430-4735-93b1-b35efd08ed8a admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:27.962276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91f2f6-8430-4735-93b1-b35efd08ed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:27.962555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91f2f6-8430-4735-93b1-b35efd08ed8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:27.963876 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91f2f6-8430-4735-93b1-b35efd08ed8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:27.965473 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91f2f6-8430-4735-93b1-b35efd08ed8a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:27.980187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1412/2835] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:27.982073 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b430d-e47a-4410-ab04-cefd8433790e admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:28.001235 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b430d-e47a-4410-ab04-cefd8433790e admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:28.009966 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389735c-2bf8-4b12-ba6d-ee45249da7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:28.011315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389735c-2bf8-4b12-ba6d-ee45249da7f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:28.011819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389735c-2bf8-4b12-ba6d-ee45249da7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:28.012309 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389735c-2bf8-4b12-ba6d-ee45249da7f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:28.013718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389735c-2bf8-4b12-ba6d-ee45249da7f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:28.017145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9389735c-2bf8-4b12-ba6d-ee45249da7f3 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:28.031625 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1424/2836] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:28.035720 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91f2f6-8430-4735-93b1-b35efd08ed8a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:28.047760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91f2f6-8430-4735-93b1-b35efd08ed8a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:28.052562 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff79b5b-83be-4f0e-bee5-3c31e23d630e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:28.053447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff79b5b-83be-4f0e-bee5-3c31e23d630e admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:28.053739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff79b5b-83be-4f0e-bee5-3c31e23d630e admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:28.054023 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff79b5b-83be-4f0e-bee5-3c31e23d630e admin admin] PATH_INFO: `/v3/projects/0be8b8329d994ec59a61a90f29b2b629/users/75c0c2f20205457cb2bf99ed549b7517/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:28.069722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff79b5b-83be-4f0e-bee5-3c31e23d630e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0be8b8329d994ec59a61a90f29b2b629, user_id=75c0c2f20205457cb2bf99ed549b7517, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:28.072481 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff79b5b-83be-4f0e-bee5-3c31e23d630e admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:28.086304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1413/2837] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:28 2022] PUT /identity/v3/projects/0be8b8329d994ec59a61a90f29b2b629/users/75c0c2f20205457cb2bf99ed549b7517/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:28.092472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d3c54-1c7a-4e0c-9ce6-34f63b1e9dfa None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:28.092781 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d3c54-1c7a-4e0c-9ce6-34f63b1e9dfa None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:28.093078 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d3c54-1c7a-4e0c-9ce6-34f63b1e9dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:28.112608 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-137d3c54-1c7a-4e0c-9ce6-34f63b1e9dfa None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:28.114015 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-137d3c54-1c7a-4e0c-9ce6-34f63b1e9dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:28.175909 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:28.188582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1425/2838] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:28.199710 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff79b5b-83be-4f0e-bee5-3c31e23d630e admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:28.201963 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-6ff79b5b-83be-4f0e-bee5-3c31e23d630e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:28.284609 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff79b5b-83be-4f0e-bee5-3c31e23d630e admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:28.288543 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426690c-80fd-4133-afa6-e4a082463b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:28.289323 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426690c-80fd-4133-afa6-e4a082463b6e service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:28.289845 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426690c-80fd-4133-afa6-e4a082463b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:28.290124 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426690c-80fd-4133-afa6-e4a082463b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:28.301403 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426690c-80fd-4133-afa6-e4a082463b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:28.302900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426690c-80fd-4133-afa6-e4a082463b6e service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:28.313900 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1414/2839] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:29.043438 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625548d8-c13a-4f97-b831-4ea6dd88e7dd None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:29.043924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625548d8-c13a-4f97-b831-4ea6dd88e7dd None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:29.044052 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625548d8-c13a-4f97-b831-4ea6dd88e7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:29.082425 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-625548d8-c13a-4f97-b831-4ea6dd88e7dd None None] MFA Rules not processed for user `75c0c2f20205457cb2bf99ed549b7517`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:29.084057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-625548d8-c13a-4f97-b831-4ea6dd88e7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:29.171256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1426/2840] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:29.183039 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426690c-80fd-4133-afa6-e4a082463b6e service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:29.184546 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-1426690c-80fd-4133-afa6-e4a082463b6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:29.252296 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426690c-80fd-4133-afa6-e4a082463b6e service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:29.256104 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c3bd2-1221-420b-8af6-8aed96234b5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:29.256831 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c3bd2-1221-420b-8af6-8aed96234b5a service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:29.257123 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c3bd2-1221-420b-8af6-8aed96234b5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:29.257399 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c3bd2-1221-420b-8af6-8aed96234b5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:29.268168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c3bd2-1221-420b-8af6-8aed96234b5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:29.269625 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c3bd2-1221-420b-8af6-8aed96234b5a service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:29.280347 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1415/2841] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:30.669415 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f51b0-8345-4062-97cd-37b0a997c492 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:30.669987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f51b0-8345-4062-97cd-37b0a997c492 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:30.670090 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f51b0-8345-4062-97cd-37b0a997c492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:30.690011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-7c4f51b0-8345-4062-97cd-37b0a997c492 None None] MFA Rules not processed for user `75c0c2f20205457cb2bf99ed549b7517`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:30.691256 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7c4f51b0-8345-4062-97cd-37b0a997c492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:30.698237 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1427/2842] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:30.810299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c3bd2-1221-420b-8af6-8aed96234b5a service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:30.813125 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-167c3bd2-1221-420b-8af6-8aed96234b5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:30.826683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:30.832491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c3bd2-1221-420b-8af6-8aed96234b5a service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:30.838489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:30.839447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:30.839734 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:30.840011 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] PATH_INFO: `/v3/users/0ad4fae1bf1a4c29af45a7f86a9b10ec` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:30.841234 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ad4fae1bf1a4c29af45a7f86a9b10ec)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:30.843198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:30.895727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ad4fae1bf1a4c29af45a7f86a9b10ec'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:30.904650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ad4fae1bf1a4c29af45a7f86a9b10ec'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:30.913425 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ad4fae1bf1a4c29af45a7f86a9b10ec'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:30.927630 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ad4fae1bf1a4c29af45a7f86a9b10ec'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:30.928363 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ad4fae1bf1a4c29af45a7f86a9b10ec'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:30.935211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1416/2843] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:30 2022] DELETE /identity/v3/users/0ad4fae1bf1a4c29af45a7f86a9b10ec => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:30.940232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3645baf-2043-4321-b1f7-fe8fe63462d5 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:30.940582 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3645baf-2043-4321-b1f7-fe8fe63462d5 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:30.940924 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3645baf-2043-4321-b1f7-fe8fe63462d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:30.961145 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-b3645baf-2043-4321-b1f7-fe8fe63462d5 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:30.962453 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b3645baf-2043-4321-b1f7-fe8fe63462d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:31.038892 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1428/2844] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:31.050269 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:31.051807 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:31.123267 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3bfea-86eb-47cb-a1e5-f87a4b713f57 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:31.133845 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406e665-60d3-4287-b8f6-dad0c727fce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:31.134818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e665-60d3-4287-b8f6-dad0c727fce1 service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:31.135117 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e665-60d3-4287-b8f6-dad0c727fce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:31.135433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5406e665-60d3-4287-b8f6-dad0c727fce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:31.147311 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406e665-60d3-4287-b8f6-dad0c727fce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:31.148690 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5406e665-60d3-4287-b8f6-dad0c727fce1 service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:31.159276 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1417/2845] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:31.536986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3645baf-2043-4321-b1f7-fe8fe63462d5 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:31.538657 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-b3645baf-2043-4321-b1f7-fe8fe63462d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:31.554980 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3645baf-2043-4321-b1f7-fe8fe63462d5 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:31.572115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:31.573559 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:31.573851 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:31.574378 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] PATH_INFO: `/v3/projects/2d04163bef8a4f83a52200c2d171c8f5` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:31.575699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d04163bef8a4f83a52200c2d171c8f5)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:31.577982 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:31.637260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d04163bef8a4f83a52200c2d171c8f5', 'request_id': 'req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:31.660645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d04163bef8a4f83a52200c2d171c8f5', 'request_id': 'req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:31.672531 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d04163bef8a4f83a52200c2d171c8f5', 'request_id': 'req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:31.688046 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] The token cache is being invalidate because project 2d04163bef8a4f83a52200c2d171c8f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:31.688404 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:31.689239 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:31.691211 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1429/2846] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:31 2022] DELETE /identity/v3/projects/2d04163bef8a4f83a52200c2d171c8f5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:31.695854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406e665-60d3-4287-b8f6-dad0c727fce1 service neutron] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:31.707895 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5406e665-60d3-4287-b8f6-dad0c727fce1 service neutron] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:31.774563 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:31.775649 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:31.776543 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:31.776919 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] PATH_INFO: `/v3/users/85b9daa53a404771a5bdbf40ea8c3315` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:31.778159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b9daa53a404771a5bdbf40ea8c3315)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:31.780147 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:31.810140 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:31.844579 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b9daa53a404771a5bdbf40ea8c3315'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:31.852503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b9daa53a404771a5bdbf40ea8c3315'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:31.862186 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b9daa53a404771a5bdbf40ea8c3315'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:31.880240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b9daa53a404771a5bdbf40ea8c3315'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:31.881079 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b9daa53a404771a5bdbf40ea8c3315'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:31.887247 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1418/2847] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:31 2022] DELETE /identity/v3/users/85b9daa53a404771a5bdbf40ea8c3315 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:31.943989 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:31.945741 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:31.957951 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:32.030380 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0263ed8-479b-4b6b-a49b-3b1af29e7d99 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:32.106418 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b0400-b09c-4406-982f-a7312fea5fff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:32.107115 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0400-b09c-4406-982f-a7312fea5fff service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:32.107420 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0400-b09c-4406-982f-a7312fea5fff service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:32.107697 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b0400-b09c-4406-982f-a7312fea5fff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:32.109537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-af0b0400-b09c-4406-982f-a7312fea5fff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:32.122665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b0400-b09c-4406-982f-a7312fea5fff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:32.124275 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b0400-b09c-4406-982f-a7312fea5fff service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:32.136013 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1430/2848] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:32.198675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:32.200719 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:32.303198 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45713fd-b7ec-44c5-b02c-b488ab1ff43b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:32.308000 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:32.308884 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:32.309173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:32.309892 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] PATH_INFO: `/v3/projects/80772998ddcd4abc8ae36b69e99a08df` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:32.310905 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] RBAC: Authorizing `identity:delete_project(project_id=80772998ddcd4abc8ae36b69e99a08df)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:32.312930 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:32.367701 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80772998ddcd4abc8ae36b69e99a08df', 'request_id': 'req-06b54296-2a91-4eed-9320-c1596d11fcba'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:32.374617 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80772998ddcd4abc8ae36b69e99a08df', 'request_id': 'req-06b54296-2a91-4eed-9320-c1596d11fcba'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:32.381087 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80772998ddcd4abc8ae36b69e99a08df', 'request_id': 'req-06b54296-2a91-4eed-9320-c1596d11fcba'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:32.394569 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] The token cache is being invalidate because project 80772998ddcd4abc8ae36b69e99a08df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:32.394986 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:32.395619 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-06b54296-2a91-4eed-9320-c1596d11fcba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:32.397168 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1419/2849] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:32 2022] DELETE /identity/v3/projects/80772998ddcd4abc8ae36b69e99a08df => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:32.404731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927ba92-720c-42c8-a2f9-2963a941cab6 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:32.405020 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927ba92-720c-42c8-a2f9-2963a941cab6 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:32.405297 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927ba92-720c-42c8-a2f9-2963a941cab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:32.425082 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a927ba92-720c-42c8-a2f9-2963a941cab6 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:32.426302 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a927ba92-720c-42c8-a2f9-2963a941cab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:32.534675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1431/2850] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:32.548109 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bfefd-70ad-4f32-86e4-2bc4cb28eb34 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:32.548388 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bfefd-70ad-4f32-86e4-2bc4cb28eb34 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:32.548560 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bfefd-70ad-4f32-86e4-2bc4cb28eb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:32.581601 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-7b0bfefd-70ad-4f32-86e4-2bc4cb28eb34 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:32.582872 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7b0bfefd-70ad-4f32-86e4-2bc4cb28eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:32.591158 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1420/2851] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:32.595744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927ba92-720c-42c8-a2f9-2963a941cab6 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:32.608931 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927ba92-720c-42c8-a2f9-2963a941cab6 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:32.611474 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938fe79-71f5-4ea2-9669-7e9197a28832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:32.612153 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938fe79-71f5-4ea2-9669-7e9197a28832 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:32.612445 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938fe79-71f5-4ea2-9669-7e9197a28832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:32.612757 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938fe79-71f5-4ea2-9669-7e9197a28832 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:32.613499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938fe79-71f5-4ea2-9669-7e9197a28832 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:39:32.613811 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938fe79-71f5-4ea2-9669-7e9197a28832 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:32.615481 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938fe79-71f5-4ea2-9669-7e9197a28832 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:32.629158 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1432/2852] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:39:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:39:32.634465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bfefd-70ad-4f32-86e4-2bc4cb28eb34 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:32.647632 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0bfefd-70ad-4f32-86e4-2bc4cb28eb34 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:32.652711 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65059017-a7da-45fa-94a7-0fee6f9d8085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:32.653381 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65059017-a7da-45fa-94a7-0fee6f9d8085 admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:32.653672 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65059017-a7da-45fa-94a7-0fee6f9d8085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:32.653954 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65059017-a7da-45fa-94a7-0fee6f9d8085 admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:32.654737 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65059017-a7da-45fa-94a7-0fee6f9d8085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:32.657025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65059017-a7da-45fa-94a7-0fee6f9d8085 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:32.680480 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:32.690372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1421/2853] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:32 2022] POST /identity/v3/projects => generated 359 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:32.695247 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938fe79-71f5-4ea2-9669-7e9197a28832 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:32.706786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938fe79-71f5-4ea2-9669-7e9197a28832 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:32.736624 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=100 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:32.773848 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d12b8d-afad-49a7-a303-cfea2d6090b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:32.774531 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d12b8d-afad-49a7-a303-cfea2d6090b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:32.774842 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d12b8d-afad-49a7-a303-cfea2d6090b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:32.775116 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d12b8d-afad-49a7-a303-cfea2d6090b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:32.775867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d12b8d-afad-49a7-a303-cfea2d6090b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:32.777832 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d12b8d-afad-49a7-a303-cfea2d6090b9 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:32.801464 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1433/2854] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:32 2022] POST /identity/v3/users => generated 280 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 08:39:32.805722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65059017-a7da-45fa-94a7-0fee6f9d8085 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:32.824982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65059017-a7da-45fa-94a7-0fee6f9d8085 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:32.834787 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d9b5b-3028-47b3-b0e7-9cafbcee6f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:32.835561 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d9b5b-3028-47b3-b0e7-9cafbcee6f3b admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:32.835854 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d9b5b-3028-47b3-b0e7-9cafbcee6f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:32.836135 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d9b5b-3028-47b3-b0e7-9cafbcee6f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:32.836916 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d9b5b-3028-47b3-b0e7-9cafbcee6f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:32.839170 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d9b5b-3028-47b3-b0e7-9cafbcee6f3b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:32.853343 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1422/2855] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:32.858126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d12b8d-afad-49a7-a303-cfea2d6090b9 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:32.871845 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d12b8d-afad-49a7-a303-cfea2d6090b9 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:32.876772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a798bf-c695-4ceb-a95a-c4128b42fce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:32.877468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a798bf-c695-4ceb-a95a-c4128b42fce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:32.877754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a798bf-c695-4ceb-a95a-c4128b42fce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:32.878042 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a798bf-c695-4ceb-a95a-c4128b42fce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:32.879786 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a798bf-c695-4ceb-a95a-c4128b42fce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:32.880533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a798bf-c695-4ceb-a95a-c4128b42fce2 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:32.897472 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1434/2856] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:32.903532 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d9b5b-3028-47b3-b0e7-9cafbcee6f3b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:32.917342 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d9b5b-3028-47b3-b0e7-9cafbcee6f3b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:32.922679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c9529-f388-4086-a69d-089a3f4a0dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:32.923592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c9529-f388-4086-a69d-089a3f4a0dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:32.923933 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c9529-f388-4086-a69d-089a3f4a0dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:32.924219 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c9529-f388-4086-a69d-089a3f4a0dc4 admin admin] PATH_INFO: `/v3/projects/8daf151499104e22ac5b4f9f7ea2b547/users/dc8d0db373d74cbc9f8d4bc615e97da1/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:32.946544 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c9529-f388-4086-a69d-089a3f4a0dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8daf151499104e22ac5b4f9f7ea2b547, user_id=dc8d0db373d74cbc9f8d4bc615e97da1, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:32.949775 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c9529-f388-4086-a69d-089a3f4a0dc4 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:32.966141 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1423/2857] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:32 2022] PUT /identity/v3/projects/8daf151499104e22ac5b4f9f7ea2b547/users/dc8d0db373d74cbc9f8d4bc615e97da1/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:39:32.981683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460b0ff-b9a5-48f5-9756-cef1b0513552 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:32.981683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460b0ff-b9a5-48f5-9756-cef1b0513552 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:32.981683 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8460b0ff-b9a5-48f5-9756-cef1b0513552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:33.020535 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-8460b0ff-b9a5-48f5-9756-cef1b0513552 None None] MFA Rules not processed for user `dc8d0db373d74cbc9f8d4bc615e97da1`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:33.021891 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-8460b0ff-b9a5-48f5-9756-cef1b0513552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:33.106255 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1435/2858] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:32 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:33.141202 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c9529-f388-4086-a69d-089a3f4a0dc4 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:33.143042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-7c7c9529-f388-4086-a69d-089a3f4a0dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:33.214018 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c9529-f388-4086-a69d-089a3f4a0dc4 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:33.218173 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02067a5-eff5-4b8f-a411-b88c6a444a9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:33.218887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02067a5-eff5-4b8f-a411-b88c6a444a9d service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:33.219174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02067a5-eff5-4b8f-a411-b88c6a444a9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:33.219465 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02067a5-eff5-4b8f-a411-b88c6a444a9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:33.230679 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02067a5-eff5-4b8f-a411-b88c6a444a9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:33.232153 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02067a5-eff5-4b8f-a411-b88c6a444a9d service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:33.242849 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1424/2859] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:33 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:33.257495 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460b0ff-b9a5-48f5-9756-cef1b0513552 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:33.340058 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8460b0ff-b9a5-48f5-9756-cef1b0513552 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:33.345547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:33.346449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:33.346770 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:33.347065 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] PATH_INFO: `/v3/users/dc8d0db373d74cbc9f8d4bc615e97da1` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:33.348355 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc8d0db373d74cbc9f8d4bc615e97da1)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:33.350457 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:33.404117 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:33.416750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc8d0db373d74cbc9f8d4bc615e97da1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:33.430318 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc8d0db373d74cbc9f8d4bc615e97da1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:33.431091 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc8d0db373d74cbc9f8d4bc615e97da1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:33.442352 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc8d0db373d74cbc9f8d4bc615e97da1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:33.455366 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc8d0db373d74cbc9f8d4bc615e97da1'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:33.461717 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1436/2860] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:33 2022] DELETE /identity/v3/users/dc8d0db373d74cbc9f8d4bc615e97da1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:33.466431 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa185-bb6d-424f-b5f0-62378849265d None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:33.466760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa185-bb6d-424f-b5f0-62378849265d None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:33.467112 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa185-bb6d-424f-b5f0-62378849265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:33.487641 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-2bdaa185-bb6d-424f-b5f0-62378849265d None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:33.488978 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2bdaa185-bb6d-424f-b5f0-62378849265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:33.602382 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1425/2861] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:33.610519 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:33.612394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:33.688954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df48aa8b-67c0-47e1-9c05-64f7c5f8b54c admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:33.692545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500414b8-e7b2-4499-887f-06d46dc86dba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:33.693229 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500414b8-e7b2-4499-887f-06d46dc86dba service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:33.693511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500414b8-e7b2-4499-887f-06d46dc86dba service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:33.693791 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500414b8-e7b2-4499-887f-06d46dc86dba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:33.705670 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500414b8-e7b2-4499-887f-06d46dc86dba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:33.707212 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500414b8-e7b2-4499-887f-06d46dc86dba service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:33.718436 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1437/2862] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:34.200240 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaa185-bb6d-424f-b5f0-62378849265d None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:34.202440 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-2bdaa185-bb6d-424f-b5f0-62378849265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:34.234415 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaa185-bb6d-424f-b5f0-62378849265d None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:34.240044 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:34.241161 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:34.241448 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:34.241728 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] PATH_INFO: `/v3/projects/8daf151499104e22ac5b4f9f7ea2b547` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:34.242982 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8daf151499104e22ac5b4f9f7ea2b547)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:34.244960 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:34.302167 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8daf151499104e22ac5b4f9f7ea2b547', 'request_id': 'req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:34.308702 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8daf151499104e22ac5b4f9f7ea2b547', 'request_id': 'req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:34.316929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8daf151499104e22ac5b4f9f7ea2b547', 'request_id': 'req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:34.319297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:34.333215 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] The token cache is being invalidate because project 8daf151499104e22ac5b4f9f7ea2b547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:34.333506 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:34.334128 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-14c856c8-d3e4-4a6f-8aa8-a9507e99b5aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:34.335728 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1426/2863] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:34 2022] DELETE /identity/v3/projects/8daf151499104e22ac5b4f9f7ea2b547 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:34.341328 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681ee04-c5cc-4e43-be9e-51b4f2aed367 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:34.341721 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681ee04-c5cc-4e43-be9e-51b4f2aed367 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:34.342074 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a681ee04-c5cc-4e43-be9e-51b4f2aed367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:34.363823 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-a681ee04-c5cc-4e43-be9e-51b4f2aed367 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:34.365220 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-a681ee04-c5cc-4e43-be9e-51b4f2aed367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:34.448649 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1438/2864] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:34.514238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67660d45-d9a7-4dee-bf33-1db78b808911 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:34.514447 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67660d45-d9a7-4dee-bf33-1db78b808911 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:34.515172 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67660d45-d9a7-4dee-bf33-1db78b808911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:34.535754 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-67660d45-d9a7-4dee-bf33-1db78b808911 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:34.537017 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-67660d45-d9a7-4dee-bf33-1db78b808911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:34.546763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1427/2865] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:34.551083 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681ee04-c5cc-4e43-be9e-51b4f2aed367 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:34.562189 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a681ee04-c5cc-4e43-be9e-51b4f2aed367 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:34.566997 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490096e-ef2a-4128-95b6-ccf725889d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:34.567671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490096e-ef2a-4128-95b6-ccf725889d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:34.567959 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490096e-ef2a-4128-95b6-ccf725889d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:34.568251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490096e-ef2a-4128-95b6-ccf725889d06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:34.569148 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490096e-ef2a-4128-95b6-ccf725889d06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:39:34.569464 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490096e-ef2a-4128-95b6-ccf725889d06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:34.571125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490096e-ef2a-4128-95b6-ccf725889d06 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:34.584290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1439/2866] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:39:34.589541 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67660d45-d9a7-4dee-bf33-1db78b808911 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:34.600211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67660d45-d9a7-4dee-bf33-1db78b808911 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:34.616659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f3874-ac79-4eca-9dd9-14701ddc6fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:34.618374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f3874-ac79-4eca-9dd9-14701ddc6fab admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:34.618374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f3874-ac79-4eca-9dd9-14701ddc6fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:34.618374 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29f3874-ac79-4eca-9dd9-14701ddc6fab admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:34.618758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f3874-ac79-4eca-9dd9-14701ddc6fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:34.621475 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29f3874-ac79-4eca-9dd9-14701ddc6fab admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:34.663869 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1428/2867] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:39:34 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:34.664693 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490096e-ef2a-4128-95b6-ccf725889d06 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:34.678711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490096e-ef2a-4128-95b6-ccf725889d06 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:34.737699 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:34.758943 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea76dc5-3aeb-4d58-a0c3-22d6849d2b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:34.759787 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea76dc5-3aeb-4d58-a0c3-22d6849d2b02 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:34.761057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea76dc5-3aeb-4d58-a0c3-22d6849d2b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:34.761057 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea76dc5-3aeb-4d58-a0c3-22d6849d2b02 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:34.761884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea76dc5-3aeb-4d58-a0c3-22d6849d2b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:34.764125 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea76dc5-3aeb-4d58-a0c3-22d6849d2b02 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:34.793434 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1440/2868] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:39:34 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:39:34.794821 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f3874-ac79-4eca-9dd9-14701ddc6fab admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:34.807187 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29f3874-ac79-4eca-9dd9-14701ddc6fab admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:34.812149 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e149951-11d1-4fd4-925d-288d624e32c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:34.812830 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e149951-11d1-4fd4-925d-288d624e32c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:34.813121 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e149951-11d1-4fd4-925d-288d624e32c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:34.813408 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e149951-11d1-4fd4-925d-288d624e32c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:34.814156 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e149951-11d1-4fd4-925d-288d624e32c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:34.815974 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e149951-11d1-4fd4-925d-288d624e32c8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:34.833674 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1429/2869] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:39:34.834754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea76dc5-3aeb-4d58-a0c3-22d6849d2b02 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:34.845574 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea76dc5-3aeb-4d58-a0c3-22d6849d2b02 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:34.850727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41fd19e-45f5-4d9c-aabe-5fcffcf7430e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:34.851406 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fd19e-45f5-4d9c-aabe-5fcffcf7430e admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:34.851695 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fd19e-45f5-4d9c-aabe-5fcffcf7430e admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:34.851992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41fd19e-45f5-4d9c-aabe-5fcffcf7430e admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:34.852738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fd19e-45f5-4d9c-aabe-5fcffcf7430e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:34.854731 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41fd19e-45f5-4d9c-aabe-5fcffcf7430e admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:34.870509 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1441/2870] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 08:39:34.873052 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e149951-11d1-4fd4-925d-288d624e32c8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:34.883855 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e149951-11d1-4fd4-925d-288d624e32c8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:34.892762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9901e72-8eb9-45ba-83f7-21f77d908e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:34.894931 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9901e72-8eb9-45ba-83f7-21f77d908e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:34.895235 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9901e72-8eb9-45ba-83f7-21f77d908e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:34.895610 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9901e72-8eb9-45ba-83f7-21f77d908e4c admin admin] PATH_INFO: `/v3/projects/7955310074774349840e125df893645c/users/7587cddd273b4a38836b7ee54096edf6/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:34.920596 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9901e72-8eb9-45ba-83f7-21f77d908e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7955310074774349840e125df893645c, user_id=7587cddd273b4a38836b7ee54096edf6, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:34.923433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9901e72-8eb9-45ba-83f7-21f77d908e4c admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:34.939924 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1430/2871] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:39:34 2022] PUT /identity/v3/projects/7955310074774349840e125df893645c/users/7587cddd273b4a38836b7ee54096edf6/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:34.950611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55142b-a21d-45e3-85ec-c535fb7551bb None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:34.950913 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55142b-a21d-45e3-85ec-c535fb7551bb None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:34.951198 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55142b-a21d-45e3-85ec-c535fb7551bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:34.984750 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-5e55142b-a21d-45e3-85ec-c535fb7551bb None None] MFA Rules not processed for user `7587cddd273b4a38836b7ee54096edf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:34.986099 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5e55142b-a21d-45e3-85ec-c535fb7551bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:35.071213 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1442/2872] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:39:35.084378 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9901e72-8eb9-45ba-83f7-21f77d908e4c admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:35.086165 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a9901e72-8eb9-45ba-83f7-21f77d908e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:35.152725 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9901e72-8eb9-45ba-83f7-21f77d908e4c admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:35.156519 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b734151-6f6b-413a-b9d9-c69f008a4a56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:35.157196 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b734151-6f6b-413a-b9d9-c69f008a4a56 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:35.157486 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b734151-6f6b-413a-b9d9-c69f008a4a56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:35.157763 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b734151-6f6b-413a-b9d9-c69f008a4a56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:35.179894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b734151-6f6b-413a-b9d9-c69f008a4a56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:35.181419 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b734151-6f6b-413a-b9d9-c69f008a4a56 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:35.192002 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1431/2873] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:37.793025 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55142b-a21d-45e3-85ec-c535fb7551bb None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:37.882379 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55142b-a21d-45e3-85ec-c535fb7551bb None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:37.887867 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:37.888919 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:37.889207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:37.889480 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] PATH_INFO: `/v3/users/7587cddd273b4a38836b7ee54096edf6` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:37.890992 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7587cddd273b4a38836b7ee54096edf6)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:37.892952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:37.958039 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7587cddd273b4a38836b7ee54096edf6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:37.964353 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:37.973906 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7587cddd273b4a38836b7ee54096edf6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:37.975207 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7587cddd273b4a38836b7ee54096edf6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:37.987734 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7587cddd273b4a38836b7ee54096edf6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:37.996391 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7587cddd273b4a38836b7ee54096edf6'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:38.008050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ced1b-f5f6-4b79-b79e-b45126813071 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:38.008538 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ced1b-f5f6-4b79-b79e-b45126813071 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:38.008665 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ced1b-f5f6-4b79-b79e-b45126813071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:38.009953 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1443/2874] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:39:37 2022] DELETE /identity/v3/users/7587cddd273b4a38836b7ee54096edf6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:38.032099 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-b40ced1b-f5f6-4b79-b79e-b45126813071 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:39:38.033344 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b40ced1b-f5f6-4b79-b79e-b45126813071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:38.103766 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1432/2875] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:39:38.114437 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:38.115868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:38.208315 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68b01f-3155-4b23-8aec-71113747fa3a admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:38.212168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23c439-4b01-407f-9176-13f527a539a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:38.212849 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23c439-4b01-407f-9176-13f527a539a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:38.213137 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23c439-4b01-407f-9176-13f527a539a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:38.213981 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23c439-4b01-407f-9176-13f527a539a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:38.223879 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23c439-4b01-407f-9176-13f527a539a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:38.225340 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23c439-4b01-407f-9176-13f527a539a4 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:38.235952 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1444/2876] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:39:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:39:38.536616 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ced1b-f5f6-4b79-b79e-b45126813071 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:39:38.538282 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-b40ced1b-f5f6-4b79-b79e-b45126813071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:39:38.563292 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ced1b-f5f6-4b79-b79e-b45126813071 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:39:38.569727 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:39:38.570670 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:39:38.570934 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:39:38.571211 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] PATH_INFO: `/v3/projects/7955310074774349840e125df893645c` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:39:38.574600 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7955310074774349840e125df893645c)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:39:38.576675 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:39:38.614148 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:INSERT=3,keystone:DELETE=2,keystone:SELECT=95 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:38.647550 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7955310074774349840e125df893645c', 'request_id': 'req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:38.657489 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7955310074774349840e125df893645c', 'request_id': 'req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:38.670673 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7955310074774349840e125df893645c', 'request_id': 'req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:38.689818 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] The token cache is being invalidate because project 7955310074774349840e125df893645c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:39:38.690108 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:38.690797 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-d00ab67a-ad89-4c3f-9a78-70c9400fa1e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:39:38.692355 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1433/2877] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:39:38 2022] DELETE /identity/v3/projects/7955310074774349840e125df893645c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:39:48.231383 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=36,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:39:48.686621 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:40:12.438727 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23c439-4b01-407f-9176-13f527a539a4 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:12.441112 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7c23c439-4b01-407f-9176-13f527a539a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:12.464789 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23c439-4b01-407f-9176-13f527a539a4 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:12.568174 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:12.569718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:12.570227 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:12.570769 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] PATH_INFO: `/v3/users/7a87e9c648e34e7c970f5cee8b8a5e06` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:12.572954 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a87e9c648e34e7c970f5cee8b8a5e06)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:12.576746 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:12.645681 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a87e9c648e34e7c970f5cee8b8a5e06'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:12.660665 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a87e9c648e34e7c970f5cee8b8a5e06'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:12.661394 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a87e9c648e34e7c970f5cee8b8a5e06'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:12.669597 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a87e9c648e34e7c970f5cee8b8a5e06'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:12.678711 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a87e9c648e34e7c970f5cee8b8a5e06'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:12.685142 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1445/2878] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:40:12 2022] DELETE /identity/v3/users/7a87e9c648e34e7c970f5cee8b8a5e06 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:40:12.690681 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6b671-766a-497c-8849-987653c2af58 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:12.691319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6b671-766a-497c-8849-987653c2af58 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:12.691319 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6b671-766a-497c-8849-987653c2af58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:12.711422 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-21b6b671-766a-497c-8849-987653c2af58 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:40:12.712661 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-21b6b671-766a-497c-8849-987653c2af58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:12.782887 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1434/2879] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:40:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:40:12.794975 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:12.796470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:12.871023 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6be3a-965a-4189-8780-3b3be80586ec admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:12.875307 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83320aad-e5c2-47fb-a112-0833c4f6858e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:12.876233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83320aad-e5c2-47fb-a112-0833c4f6858e service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:12.876776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83320aad-e5c2-47fb-a112-0833c4f6858e service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:12.876776 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83320aad-e5c2-47fb-a112-0833c4f6858e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:12.888334 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83320aad-e5c2-47fb-a112-0833c4f6858e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:12.891359 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83320aad-e5c2-47fb-a112-0833c4f6858e service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:12.905051 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1446/2880] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:40:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:40:13.334566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6b671-766a-497c-8849-987653c2af58 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:13.336056 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-21b6b671-766a-497c-8849-987653c2af58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:13.366810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6b671-766a-497c-8849-987653c2af58 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:13.372929 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:13.374315 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:13.374750 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:13.376765 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] PATH_INFO: `/v3/projects/45afcd5f1d304b6e8f8b3b44d61f2b97` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:13.378250 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45afcd5f1d304b6e8f8b3b44d61f2b97)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:13.380484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:13.447533 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45afcd5f1d304b6e8f8b3b44d61f2b97', 'request_id': 'req-a351d6ec-771f-486c-bb4b-c3c522d2fa48'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:13.455762 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45afcd5f1d304b6e8f8b3b44d61f2b97', 'request_id': 'req-a351d6ec-771f-486c-bb4b-c3c522d2fa48'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:13.467346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45afcd5f1d304b6e8f8b3b44d61f2b97', 'request_id': 'req-a351d6ec-771f-486c-bb4b-c3c522d2fa48'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:13.491401 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] The token cache is being invalidate because project 45afcd5f1d304b6e8f8b3b44d61f2b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:40:13.491932 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:13.493043 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-a351d6ec-771f-486c-bb4b-c3c522d2fa48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:13.495852 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1435/2881] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:40:13 2022] DELETE /identity/v3/projects/45afcd5f1d304b6e8f8b3b44d61f2b97 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 08:40:13.502871 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cc444-fd16-4ff2-b631-449c4f7b14d4 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:13.503470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cc444-fd16-4ff2-b631-449c4f7b14d4 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:13.503899 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cc444-fd16-4ff2-b631-449c4f7b14d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:13.535185 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-cd3cc444-fd16-4ff2-b631-449c4f7b14d4 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:40:13.537353 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-cd3cc444-fd16-4ff2-b631-449c4f7b14d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:13.569690 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:40:13.625263 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1447/2882] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:40:13.635860 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c19540-1b9f-490a-94dc-3f9717a7a5c2 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:13.636252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c19540-1b9f-490a-94dc-3f9717a7a5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:13.636555 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c19540-1b9f-490a-94dc-3f9717a7a5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:13.660660 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-a2c19540-1b9f-490a-94dc-3f9717a7a5c2 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:40:13.661993 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-a2c19540-1b9f-490a-94dc-3f9717a7a5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:13.671453 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1436/2883] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:40:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:40:13.679035 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cc444-fd16-4ff2-b631-449c4f7b14d4 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:13.691843 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cc444-fd16-4ff2-b631-449c4f7b14d4 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:13.696742 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaab2b-686a-4b6f-9a3e-819039960166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:13.697537 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaab2b-686a-4b6f-9a3e-819039960166 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:13.697988 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaab2b-686a-4b6f-9a3e-819039960166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:13.698468 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aaab2b-686a-4b6f-9a3e-819039960166 admin admin] PATH_INFO: `/v3/domains` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:13.699722 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaab2b-686a-4b6f-9a3e-819039960166 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80226) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 08:40:13.700225 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaab2b-686a-4b6f-9a3e-819039960166 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:13.703671 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aaab2b-686a-4b6f-9a3e-819039960166 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:13.724444 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1448/2884] 10.222.0.64 () {58 vars in 1152 bytes} [Wed Jul 20 08:40:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 08:40:13.729383 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c19540-1b9f-490a-94dc-3f9717a7a5c2 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:13.745174 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c19540-1b9f-490a-94dc-3f9717a7a5c2 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:13.755071 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2837e-130f-4891-8d8a-8333f215412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:13.756409 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2837e-130f-4891-8d8a-8333f215412a admin admin] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:13.757010 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2837e-130f-4891-8d8a-8333f215412a admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:13.757592 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2837e-130f-4891-8d8a-8333f215412a admin admin] PATH_INFO: `/v3/projects` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:13.759218 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2837e-130f-4891-8d8a-8333f215412a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:13.763760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2837e-130f-4891-8d8a-8333f215412a admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:13.824360 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1437/2885] 10.222.0.64 () {60 vars in 1152 bytes} [Wed Jul 20 08:40:13 2022] POST /identity/v3/projects => generated 391 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:40:13.829884 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaab2b-686a-4b6f-9a3e-819039960166 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:13.844558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aaab2b-686a-4b6f-9a3e-819039960166 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:13.927060 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832cc7c-4ea1-4e03-9845-81860ff635b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:13.927898 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832cc7c-4ea1-4e03-9845-81860ff635b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:13.928519 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832cc7c-4ea1-4e03-9845-81860ff635b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:13.928986 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832cc7c-4ea1-4e03-9845-81860ff635b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:13.929868 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832cc7c-4ea1-4e03-9845-81860ff635b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:13.932273 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832cc7c-4ea1-4e03-9845-81860ff635b1 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:13.959159 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1449/2886] 10.222.0.64 () {60 vars in 1143 bytes} [Wed Jul 20 08:40:13 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 08:40:13.963551 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2837e-130f-4891-8d8a-8333f215412a admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:13.975218 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2837e-130f-4891-8d8a-8333f215412a admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:13.980061 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da1613-9d77-4295-b6cc-bad0925ff012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:13.980744 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da1613-9d77-4295-b6cc-bad0925ff012 admin admin] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:13.981032 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da1613-9d77-4295-b6cc-bad0925ff012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:13.981346 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da1613-9d77-4295-b6cc-bad0925ff012 admin admin] PATH_INFO: `/v3/roles` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:13.982094 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da1613-9d77-4295-b6cc-bad0925ff012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:13.983760 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da1613-9d77-4295-b6cc-bad0925ff012 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:13.996470 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1438/2887] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:40:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:40:14.001123 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832cc7c-4ea1-4e03-9845-81860ff635b1 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:14.012874 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832cc7c-4ea1-4e03-9845-81860ff635b1 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:14.018511 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f7b93-34dd-4d7f-aaa0-6f59cc64b744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:14.019295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f7b93-34dd-4d7f-aaa0-6f59cc64b744 admin admin] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:14.019581 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f7b93-34dd-4d7f-aaa0-6f59cc64b744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:14.019857 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f7b93-34dd-4d7f-aaa0-6f59cc64b744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:14.020607 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f7b93-34dd-4d7f-aaa0-6f59cc64b744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:14.022275 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f7b93-34dd-4d7f-aaa0-6f59cc64b744 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:14.034308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1450/2888] 10.222.0.64 () {58 vars in 1121 bytes} [Wed Jul 20 08:40:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 08:40:14.039037 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da1613-9d77-4295-b6cc-bad0925ff012 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:14.051548 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da1613-9d77-4295-b6cc-bad0925ff012 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:14.057107 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bd7b2-f229-4696-acef-ecffe71a900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:14.058080 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bd7b2-f229-4696-acef-ecffe71a900d admin admin] REQUEST_METHOD: `PUT` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:14.058411 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bd7b2-f229-4696-acef-ecffe71a900d admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:14.058783 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3bd7b2-f229-4696-acef-ecffe71a900d admin admin] PATH_INFO: `/v3/projects/a69dfcb9eba34b82a15ee739cc5cb761/users/a7a156c0e47b4ada922809c9912c4025/roles/ef48ab9c5b844be59351a260130d3071` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:14.076096 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bd7b2-f229-4696-acef-ecffe71a900d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a69dfcb9eba34b82a15ee739cc5cb761, user_id=a7a156c0e47b4ada922809c9912c4025, role_id=ef48ab9c5b844be59351a260130d3071)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:14.079021 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3bd7b2-f229-4696-acef-ecffe71a900d admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:14.094020 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1439/2889] 10.222.0.64 () {60 vars in 1482 bytes} [Wed Jul 20 08:40:14 2022] PUT /identity/v3/projects/a69dfcb9eba34b82a15ee739cc5cb761/users/a7a156c0e47b4ada922809c9912c4025/roles/ef48ab9c5b844be59351a260130d3071 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:40:14.106094 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c82ac6-da52-44ba-a29a-1fe90c8b78b3 None None] REQUEST_METHOD: `POST` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:14.106426 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c82ac6-da52-44ba-a29a-1fe90c8b78b3 None None] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:14.106806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c82ac6-da52-44ba-a29a-1fe90c8b78b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:14.141545 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.auth.core [None req-d7c82ac6-da52-44ba-a29a-1fe90c8b78b3 None None] MFA Rules not processed for user `a7a156c0e47b4ada922809c9912c4025`. Rule list: `[]` (Enabled: `True`). {{(pid=80226) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:40:14.142882 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d7c82ac6-da52-44ba-a29a-1fe90c8b78b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:14.218614 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1451/2890] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:40:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:40:14.230863 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bd7b2-f229-4696-acef-ecffe71a900d admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:14.232901 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-5a3bd7b2-f229-4696-acef-ecffe71a900d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:14.259239 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:40:14.309530 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3bd7b2-f229-4696-acef-ecffe71a900d admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:14.314650 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f3e35-88ed-4c2d-ab4a-5717810db9b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '41a647bd72f344fa963dcd3eb5434b6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:14.315657 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3e35-88ed-4c2d-ab4a-5717810db9b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:14.316077 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3e35-88ed-4c2d-ab4a-5717810db9b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:14.316488 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3e35-88ed-4c2d-ab4a-5717810db9b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:14.327573 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f3e35-88ed-4c2d-ab4a-5717810db9b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:14.330183 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f3e35-88ed-4c2d-ab4a-5717810db9b3 service cinder] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:14.343640 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1440/2891] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:40:14 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:40:24.215075 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:40:24.338159 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=17 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:40:43.906744 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c82ac6-da52-44ba-a29a-1fe90c8b78b3 None None] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:43.908470 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-d7c82ac6-da52-44ba-a29a-1fe90c8b78b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:43.976651 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c82ac6-da52-44ba-a29a-1fe90c8b78b3 None None] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:43.980613 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b304712-886c-46a2-bd7b-0ba64b0ab568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:43.981295 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b304712-886c-46a2-bd7b-0ba64b0ab568 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:43.981579 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b304712-886c-46a2-bd7b-0ba64b0ab568 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:43.981854 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b304712-886c-46a2-bd7b-0ba64b0ab568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:43.984519 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-7b304712-886c-46a2-bd7b-0ba64b0ab568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:44.084345 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b304712-886c-46a2-bd7b-0ba64b0ab568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:44.086192 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b304712-886c-46a2-bd7b-0ba64b0ab568 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:44.098893 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1452/2892] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:40:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:40:45.284686 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f3e35-88ed-4c2d-ab4a-5717810db9b3 service cinder] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:45.286722 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-282f3e35-88ed-4c2d-ab4a-5717810db9b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:45.299491 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f3e35-88ed-4c2d-ab4a-5717810db9b3 service cinder] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:45.303055 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0af5a79-d56e-4a34-97ce-d55e632a797c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:45.303959 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af5a79-d56e-4a34-97ce-d55e632a797c service neutron] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:45.304246 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af5a79-d56e-4a34-97ce-d55e632a797c service neutron] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:45.304528 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0af5a79-d56e-4a34-97ce-d55e632a797c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:45.306980 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c0af5a79-d56e-4a34-97ce-d55e632a797c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:45.320597 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af5a79-d56e-4a34-97ce-d55e632a797c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:45.322050 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0af5a79-d56e-4a34-97ce-d55e632a797c service neutron] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:45.333299 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1441/2893] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:40:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:40:48.910547 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b304712-886c-46a2-bd7b-0ba64b0ab568 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:48.982323 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b304712-886c-46a2-bd7b-0ba64b0ab568 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:48.987844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:48.988730 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:48.989018 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:48.989297 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] PATH_INFO: `/v3/users/a7a156c0e47b4ada922809c9912c4025` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:48.990499 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a156c0e47b4ada922809c9912c4025)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:48.992510 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:49.044456 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:40:49.048240 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a156c0e47b4ada922809c9912c4025'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:49.061675 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a156c0e47b4ada922809c9912c4025'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:49.062556 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a156c0e47b4ada922809c9912c4025'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:49.070833 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a156c0e47b4ada922809c9912c4025'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:49.078502 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a156c0e47b4ada922809c9912c4025'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:49.084783 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1453/2894] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:40:48 2022] DELETE /identity/v3/users/a7a156c0e47b4ada922809c9912c4025 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:40:49.089376 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20a4c4-9dbd-44e2-8f8c-68838c651a41 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:49.089927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20a4c4-9dbd-44e2-8f8c-68838c651a41 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:49.089927 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20a4c4-9dbd-44e2-8f8c-68838c651a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:49.110220 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-fd20a4c4-9dbd-44e2-8f8c-68838c651a41 None None] MFA Rules not processed for user `c5ec92316a2c452dab52aa49d7256f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:40:49.111487 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fd20a4c4-9dbd-44e2-8f8c-68838c651a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:49.171906 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1442/2895] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 08:40:49.181611 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:49.183136 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:49.242234 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db88e7-51fc-4222-af0e-f674839eacda admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:49.246098 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286cc80-3ba0-4d35-a697-a1f20346df67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:49.246799 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286cc80-3ba0-4d35-a697-a1f20346df67 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:49.247086 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286cc80-3ba0-4d35-a697-a1f20346df67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:49.247517 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286cc80-3ba0-4d35-a697-a1f20346df67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:49.257368 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286cc80-3ba0-4d35-a697-a1f20346df67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:49.258686 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286cc80-3ba0-4d35-a697-a1f20346df67 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:49.269752 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1454/2896] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:40:49.567252 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20a4c4-9dbd-44e2-8f8c-68838c651a41 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:49.568844 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fd20a4c4-9dbd-44e2-8f8c-68838c651a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:49.583591 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20a4c4-9dbd-44e2-8f8c-68838c651a41 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:49.589191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:49.590857 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:49.592114 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:49.592426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] PATH_INFO: `/v3/projects/a69dfcb9eba34b82a15ee739cc5cb761` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:49.594755 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a69dfcb9eba34b82a15ee739cc5cb761)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:49.596718 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:49.648304 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a69dfcb9eba34b82a15ee739cc5cb761', 'request_id': 'req-42332d80-4f28-4851-ab44-66dce214b06b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:49.654132 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a69dfcb9eba34b82a15ee739cc5cb761', 'request_id': 'req-42332d80-4f28-4851-ab44-66dce214b06b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:49.660332 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a69dfcb9eba34b82a15ee739cc5cb761', 'request_id': 'req-42332d80-4f28-4851-ab44-66dce214b06b'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:49.674427 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] The token cache is being invalidate because project a69dfcb9eba34b82a15ee739cc5cb761 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80225) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:40:49.674688 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:49.675414 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:40:49.676992 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1443/2897] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:40:49 2022] DELETE /identity/v3/projects/a69dfcb9eba34b82a15ee739cc5cb761 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:40:56.343850 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286cc80-3ba0-4d35-a697-a1f20346df67 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:40:56.345586 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-4286cc80-3ba0-4d35-a697-a1f20346df67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:56.357622 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286cc80-3ba0-4d35-a697-a1f20346df67 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:40:56.409645 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda479b2-ecd8-456c-abe2-2a5adb1ba433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3b03383554346138b98becb2828851a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:40:56.410282 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda479b2-ecd8-456c-abe2-2a5adb1ba433 service neutron] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:40:56.410561 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda479b2-ecd8-456c-abe2-2a5adb1ba433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:40:56.410864 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda479b2-ecd8-456c-abe2-2a5adb1ba433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:40:56.412656 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dda479b2-ecd8-456c-abe2-2a5adb1ba433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:40:56.605585 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda479b2-ecd8-456c-abe2-2a5adb1ba433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:40:56.607092 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda479b2-ecd8-456c-abe2-2a5adb1ba433 service neutron] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:40:56.618089 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1455/2898] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:40:59.673810 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:41:00.055683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:41:00.059394 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:41:00.182022 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42332d80-4f28-4851-ab44-66dce214b06b admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:41:00.189646 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ac37e-5886-406c-8d34-259cc27648c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:41:00.190665 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ac37e-5886-406c-8d34-259cc27648c4 service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:41:00.190870 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ac37e-5886-406c-8d34-259cc27648c4 service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:41:00.191221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73ac37e-5886-406c-8d34-259cc27648c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:41:00.194119 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f73ac37e-5886-406c-8d34-259cc27648c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:41:00.208742 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ac37e-5886-406c-8d34-259cc27648c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:41:00.210484 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73ac37e-5886-406c-8d34-259cc27648c4 service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:41:00.231731 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1444/2899] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:41:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:41:03.795987 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda479b2-ecd8-456c-abe2-2a5adb1ba433 service neutron] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:41:03.797638 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-dda479b2-ecd8-456c-abe2-2a5adb1ba433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:41:03.813308 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda479b2-ecd8-456c-abe2-2a5adb1ba433 service neutron] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:41:03.820890 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26480bba-7c5e-4628-97f2-2dceaa5ed622 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:41:03.821589 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26480bba-7c5e-4628-97f2-2dceaa5ed622 service placement] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:41:03.821872 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26480bba-7c5e-4628-97f2-2dceaa5ed622 service placement] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:41:03.822224 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26480bba-7c5e-4628-97f2-2dceaa5ed622 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:41:03.824118 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-26480bba-7c5e-4628-97f2-2dceaa5ed622 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:41:03.837551 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26480bba-7c5e-4628-97f2-2dceaa5ed622 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:41:03.839251 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26480bba-7c5e-4628-97f2-2dceaa5ed622 service placement] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:41:03.849880 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:41:03.865558 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1456/2900] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:41:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:41:03.879558 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ac37e-5886-406c-8d34-259cc27648c4 service placement] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:41:03.881352 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-f73ac37e-5886-406c-8d34-259cc27648c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:41:03.902329 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73ac37e-5886-406c-8d34-259cc27648c4 service placement] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:41:03.907743 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:41:03.908696 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:41:03.909238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:41:03.909238 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] PATH_INFO: `/v3/users/19024dea6ca94745b38158beb08f29fc` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:41:03.926075 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19024dea6ca94745b38158beb08f29fc)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:41:03.928086 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:41:03.975879 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19024dea6ca94745b38158beb08f29fc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:03.982893 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19024dea6ca94745b38158beb08f29fc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:03.991643 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19024dea6ca94745b38158beb08f29fc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.005758 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19024dea6ca94745b38158beb08f29fc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.006503 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19024dea6ca94745b38158beb08f29fc'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.015908 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1445/2901] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:41:03 2022] DELETE /identity/v3/users/19024dea6ca94745b38158beb08f29fc => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:41:04.205779 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26480bba-7c5e-4628-97f2-2dceaa5ed622 service placement] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:41:04.217541 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26480bba-7c5e-4628-97f2-2dceaa5ed622 service placement] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:41:04.279646 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:41:04.281151 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:41:04.281432 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:41:04.281664 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] PATH_INFO: `/v3/projects/fcfc8ebb18c54de29556d0b93593d676` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:41:04.293269 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcfc8ebb18c54de29556d0b93593d676)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:41:04.295548 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:41:04.346806 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcfc8ebb18c54de29556d0b93593d676', 'request_id': 'req-ed158310-e7ab-4f20-9312-ee9a334d6a66'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.355243 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcfc8ebb18c54de29556d0b93593d676', 'request_id': 'req-ed158310-e7ab-4f20-9312-ee9a334d6a66'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.362828 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcfc8ebb18c54de29556d0b93593d676', 'request_id': 'req-ed158310-e7ab-4f20-9312-ee9a334d6a66'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.374870 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] The token cache is being invalidate because project fcfc8ebb18c54de29556d0b93593d676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:41:04.375449 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.375819 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.377397 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1457/2902] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:41:04 2022] DELETE /identity/v3/projects/fcfc8ebb18c54de29556d0b93593d676 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:41:04.381450 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:41:04.382981 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:41:04.397264 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef40a7-a7fb-4679-9295-71e7e8333da8 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:41:04.447659 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:41:04.462064 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34712bd3-e96a-49ec-83d7-dde45de6e660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:41:04.463014 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34712bd3-e96a-49ec-83d7-dde45de6e660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:41:04.463302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34712bd3-e96a-49ec-83d7-dde45de6e660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:41:04.463581 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34712bd3-e96a-49ec-83d7-dde45de6e660 admin admin] PATH_INFO: `/v3/users/b1406957b934446f924f32436d597461` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:41:04.478312 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34712bd3-e96a-49ec-83d7-dde45de6e660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1406957b934446f924f32436d597461)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:41:04.480400 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34712bd3-e96a-49ec-83d7-dde45de6e660 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:41:04.529515 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-34712bd3-e96a-49ec-83d7-dde45de6e660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1406957b934446f924f32436d597461'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.537739 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-34712bd3-e96a-49ec-83d7-dde45de6e660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1406957b934446f924f32436d597461'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.545251 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-34712bd3-e96a-49ec-83d7-dde45de6e660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1406957b934446f924f32436d597461'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.559191 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-34712bd3-e96a-49ec-83d7-dde45de6e660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1406957b934446f924f32436d597461'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.559915 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-34712bd3-e96a-49ec-83d7-dde45de6e660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1406957b934446f924f32436d597461'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.566184 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1446/2903] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:41:04 2022] DELETE /identity/v3/users/b1406957b934446f924f32436d597461 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:41:04.746126 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:41:04.747701 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:41:04.766290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed158310-e7ab-4f20-9312-ee9a334d6a66 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:41:04.836772 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:41:04.837677 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:41:04.837958 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:41:04.838327 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] PATH_INFO: `/v3/projects/0b42df36a52d4b21b94ccd2d90d1e7a6` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:41:04.852202 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b42df36a52d4b21b94ccd2d90d1e7a6)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:41:04.854982 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:41:04.881929 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:41:04.928514 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b42df36a52d4b21b94ccd2d90d1e7a6', 'request_id': 'req-38c3c28a-3fa1-4806-954e-f3f950867478'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.938604 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b42df36a52d4b21b94ccd2d90d1e7a6', 'request_id': 'req-38c3c28a-3fa1-4806-954e-f3f950867478'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.947232 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b42df36a52d4b21b94ccd2d90d1e7a6', 'request_id': 'req-38c3c28a-3fa1-4806-954e-f3f950867478'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.960738 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] The token cache is being invalidate because project 0b42df36a52d4b21b94ccd2d90d1e7a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:41:04.961027 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.961718 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:41:04.963326 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1458/2904] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:41:04 2022] DELETE /identity/v3/projects/0b42df36a52d4b21b94ccd2d90d1e7a6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:41:11.686426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d645a-be9b-4815-b26e-95a44ef756f0 None None] REQUEST_METHOD: `POST` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:41:11.686972 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d645a-be9b-4815-b26e-95a44ef756f0 None None] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:41:11.687097 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d645a-be9b-4815-b26e-95a44ef756f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:41:11.704890 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.auth.core [None req-221d645a-be9b-4815-b26e-95a44ef756f0 None None] MFA Rules not processed for user `75c0c2f20205457cb2bf99ed549b7517`. Rule list: `[]` (Enabled: `True`). {{(pid=80225) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 08:41:11.706025 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-221d645a-be9b-4815-b26e-95a44ef756f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:41:11.761502 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1447/2905] 10.222.0.64 () {58 vars in 957 bytes} [Wed Jul 20 08:41:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 08:41:14.956219 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=18,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:41:21.754848 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=60,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:41:49.314816 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:41:49.317011 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:41:49.405575 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c3c28a-3fa1-4806-954e-f3f950867478 admin admin] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:41:49.411290 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a3b83-57ae-4d12-8bf7-e9f39664cf5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '84f0da8709624834976d0662aeb02ea6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:41:49.411976 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a3b83-57ae-4d12-8bf7-e9f39664cf5a service nova] REQUEST_METHOD: `GET` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:41:49.412260 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a3b83-57ae-4d12-8bf7-e9f39664cf5a service nova] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:41:49.412533 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a3b83-57ae-4d12-8bf7-e9f39664cf5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:41:49.414551 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.fernet_utils [None req-bc0a3b83-57ae-4d12-8bf7-e9f39664cf5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80226) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:41:49.428068 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a3b83-57ae-4d12-8bf7-e9f39664cf5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:41:49.429669 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a3b83-57ae-4d12-8bf7-e9f39664cf5a service nova] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:41:49.441668 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1459/2906] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:41:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:41:59.435934 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=34 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:42:15.275556 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d645a-be9b-4815-b26e-95a44ef756f0 None None] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:42:15.277205 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-221d645a-be9b-4815-b26e-95a44ef756f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:42:15.344100 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d645a-be9b-4815-b26e-95a44ef756f0 None None] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:42:15.348612 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:42:15.349485 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:42:15.349769 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:42:15.350042 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] PATH_INFO: `/v3/users/75c0c2f20205457cb2bf99ed549b7517` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:42:15.351221 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75c0c2f20205457cb2bf99ed549b7517)` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:42:15.353155 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:42:15.401053 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75c0c2f20205457cb2bf99ed549b7517'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:42:15.407997 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75c0c2f20205457cb2bf99ed549b7517'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:42:15.415302 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75c0c2f20205457cb2bf99ed549b7517'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:42:15.428297 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75c0c2f20205457cb2bf99ed549b7517'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:42:15.428958 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.notifications [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75c0c2f20205457cb2bf99ed549b7517'} {{(pid=80225) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:42:15.434957 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1448/2907] 10.222.0.64 () {58 vars in 1223 bytes} [Wed Jul 20 08:42:15 2022] DELETE /identity/v3/users/75c0c2f20205457cb2bf99ed549b7517 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:42:15.610587 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a3b83-57ae-4d12-8bf7-e9f39664cf5a service nova] Authenticating user token {{(pid=80226) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:42:15.621384 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a3b83-57ae-4d12-8bf7-e9f39664cf5a service nova] Validating token access rules against request {{(pid=80226) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:42:15.682357 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c5ec92316a2c452dab52aa49d7256f2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '10580c2140c64e24a4fb1ff11f27a985', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80226) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:42:15.683168 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:42:15.683451 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:42:15.683726 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] PATH_INFO: `/v3/projects/0be8b8329d994ec59a61a90f29b2b629` {{(pid=80226) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:42:15.684904 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0be8b8329d994ec59a61a90f29b2b629)` {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:42:15.686844 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] RBAC: Authorization granted {{(pid=80226) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:42:15.734754 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0be8b8329d994ec59a61a90f29b2b629', 'request_id': 'req-3ef5e0ef-37ab-4e32-9266-bd18606662d8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:42:15.743267 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0be8b8329d994ec59a61a90f29b2b629', 'request_id': 'req-3ef5e0ef-37ab-4e32-9266-bd18606662d8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:42:15.749640 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0be8b8329d994ec59a61a90f29b2b629', 'request_id': 'req-3ef5e0ef-37ab-4e32-9266-bd18606662d8'} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:42:15.760663 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] The token cache is being invalidate because project 0be8b8329d994ec59a61a90f29b2b629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80226) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 08:42:15.760948 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:42:15.761676 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG keystone.notifications [None req-3ef5e0ef-37ab-4e32-9266-bd18606662d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80226) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 08:42:15.763233 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: [pid: 80226|app: 0|req: 1460/2908] 10.222.0.64 () {58 vars in 1232 bytes} [Wed Jul 20 08:42:15 2022] DELETE /identity/v3/projects/0be8b8329d994ec59a61a90f29b2b629 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 08:42:25.431426 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:42:25.761291 dv-cinder-823089-3-iscsi devstack@keystone.service[80226]: DEBUG dbcounter [-] [80226] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80226) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 08:44:47.764894 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] Authenticating user token {{(pid=80225) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 08:44:47.766372 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:44:47.906595 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32cbae-4b76-494d-b309-2f189d2ff287 admin admin] Validating token access rules against request {{(pid=80225) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 08:44:47.967139 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac192192-1764-43b1-99f9-eee5c2be9a9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '928054a577d14522bd0f00417f4f8c1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9a4d252038c649faadd6e91924ffb297', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80225) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 08:44:47.967989 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac192192-1764-43b1-99f9-eee5c2be9a9c service placement] REQUEST_METHOD: `GET` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 08:44:47.968305 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac192192-1764-43b1-99f9-eee5c2be9a9c service placement] SCRIPT_NAME: `/identity` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 08:44:47.968566 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac192192-1764-43b1-99f9-eee5c2be9a9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80225) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 08:44:47.970436 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.fernet_utils [None req-ac192192-1764-43b1-99f9-eee5c2be9a9c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80225) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 08:44:47.981811 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac192192-1764-43b1-99f9-eee5c2be9a9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 08:44:47.983683 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac192192-1764-43b1-99f9-eee5c2be9a9c service placement] RBAC: Authorization granted {{(pid=80225) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 08:44:47.993689 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: [pid: 80225|app: 0|req: 1449/2909] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Jul 20 08:44:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 08:44:57.989433 dv-cinder-823089-3-iscsi devstack@keystone.service[80225]: DEBUG dbcounter [-] [80225] Writing DB stats keystone:SELECT=89 {{(pid=80225) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}